Calendar Load Testing

Testing Process

To perform the load testing we used the Apache HTTP server benchmarking tool, also known as ab. This tool is distributed with Apache and its documentation can be found here http://httpd.apache.org/docs/2.2/programs/ab.html.

We tested version 2.0.2 of Kinetic Calendar.

We tested using the following architectures: a single tomcat instance, two load balanced tomcat instances, and four load balanced tomcat instances. Each tomcat instance was installed on its own virtual machine with the following specifications: Windows 2008 RC2, 64bit, 4GB Ram, 2x 2.4GHz, Java 7u21 (64bit), Tomcat 7.0.39 (64bit) 1GB.

In addition to testing with multiple architectures we also performed testing with a variety of calendars. Because there is processing of events on the server-side before they are sent to the client, calendars that contain more events will take more time for the application to process.

Finally, to simulate different amounts of load on the application we varied the concurrency value of ab. Raising the concurrency value simulates higher load on the application.

Additional details about the environment. We used HAProxy on Ubuntu (2x2.4GHz) as a load balancer and a single Remedy 7.6 server (4x2.4 GHZ). The single remedy server did not appear to be a bottleneck for any of the tests.

Note About Concurrency

Concurrency in these test cases does not mean number of users. Concurrency in these test cases means the number of requests being handled by the application at the same point in time. You can think of it as a number of users that click a calendar link within the same second or less.

Results

The results are included below, there is a graph and a table for each type of calendar tested.

In the tables, the value along the top is the concurrency value given ab for that test case and the value along the left was the number of tomcat instances. The value in the table is the average response time in milliseconds for that test case.

In the graphs, the value along the x-axis is the concurrency value and the value along the y-axis is the response time in milliseconds. Note that there is a line for each of the server setups.

Small Calendar

In this test case the calendar contains 25 events.

Medium Calendar

In this test case the calendar contains 100 events.

Big Calendar

In this test case the calendar contains 1000 events.

Multi Calendar

In this test case the calendar contains 100 events, but there are 4 event types. This means the calendar application must make 4 separate queries to retrieve its data. The purpose was to see the performance impact of this case to retrieving 100 events from a single source (Medium Calendar).