System tests

The aim of this series of system tests was to investigate how the AstroGrid services behaved under a continuous loading.

The tests were designed to start with a small load, gradually increase it by adding more test clients, and then reduce it back down to zero.

The python script used in the tests sends an ADQL query to the local DSA and asked it to store the results in the local MySpace service. The python script waits for the job to complete, and then sends the same query again, repeating the loop for a set number of times.

The stress test involved running multiple copies of the python script on a number of client machines, all sending queries to the same DSA and MySpace services.

The test infrastructure was setup to start with one test client sending requests, and then gradually add more and more test clients at 5 minute intervals, gradually increasing the load on the services. The python scripts run for a set number of loops, and then exit, gradually decreasing the load on the services as each test client finishes.

The primary output from the tests logs the time taken to perform each query and store the results. As the load increases the query execution time should increase at a linear rate, as the DSA limits the number of active threads and puts new queries into a queue. As the load decreases, the execution time should come back down to the initial figure for the unloaded service.

Service configuration

For the initial set of tests, each service was installed on a separate machine. The machines chose to run the tests were relatively low specification machines with limited resources in order to see how the services responded when close to the resource limits of the host machine.

Running the services on a production machine with multiple cpus and giga bytes of memory would make it more difficult to push the services to the limit of the available resources. Running the services on low specification machines allows us to see how they behave when pushed close to the limit of the system resources.

Individual test runs

• SystemTests20070617-00
• SystemTests20070618-00
• SystemTests20070618-01
• SystemTests20070618-02
• SystemTests20070619-00