Performance Testing Blog

Here it begins - my performance testing blog!
I'm going to try to update this frequently - whether that's monthly (at least) or weekly yet remain s to be seen.
It should be (I hope) interesting and relevant and where I get permission from customers and partners I'll include some case studies and short stories.

So let's start with my 1st entry: What is performance testing?
We can quote ISEB’s definitions, what we find in Wikipedia or what the current buzzword using conference speaker is saying which are all valuable (perhaps not so much with the latter), but where would be the value in repeating already stated definitions? I’m going to start with what I see as the definition of performance testing & you can decide if it’s valuable or not.

Performance Testing from my experience 90% of the time refers to evaluating the performance of applications deployed via web / HTTP technology. The other 10% of cases may be deployed via other protocols such as Citrix (ICA), Oracle (OCI), Oracle Forms (OCX?), SAP (proprietary), Remote Desktop (RDP) & bespoke (usually some for of TCP / socket connection) and many more.
Either way in all cases the application whose performance is to be measured is critical to the business, in fact it’s usually a core application envisaged to be used every day at high volume. Often these are internal applications such as financial reporting software, content management systems, timesheet systems and so on. If not internal applications the company itself may in reality be the web site application, take the example of an on line betting company, or a trading company like ebay. The companies performance (in $/£/€ terms) is directly related to the performance of the web site. Alternatively the company may rely solely on the web site as it’s point of contact with the public, take an events company – the web site may be used to register athletes for a run, it may be used to sell tickets for a concert.
So if performance testing is so closely related to business performance what is it we really need to measure? Well if we go back to what business’ main goals are, they relate to transactions – business transactions. So in fact most of the time the application under test has to support a given transaction rate to be able to support the business. It’s not as simple as that though as the transaction rate has to be maintained while a certain number of real users are using the application, additionally the application has to remain stable (up and running) and usable (i.e. not tortoise / snail like).
So take a timesheet application for example, end of the week or month a global company has 10,000 employees all needing to submit timesheets. Human nature dictates that people generally leave this to the last minute. However due to time zones there is a 24 hour windows for the timesheets to be submitted and the biggest country has 4,000 employees. Worst case scenario is that at 5pm that office decides to submit all timesheets in that hour, this means the application will need to deal with 4000 timesheet submissions in 1 hour, if you have the luxury of having historic data (perhaps from a previous system) you may know that you typically see no more than 400 users accessing the system at one time.
This means we can say that we need to simulate 400 concurrent users and run the test for at least an hour at a timesheet submission rate of 10,000 per hour.
What about public web sites where access is simply not controlled – that’s pretty difficult but I’ve heard stories about how companies deal with that.
This entry’s pretty long now so I’ll talk about that next time.