DougHughes.net

I was recently confronted with an application that suffered from abysmal performance… and by “abysmal” I mean “really really bad”. My task was simple: Make the website go fast! It was an interesting challenge, really, and it got me thinking about the difference between server performance and application performance. All tools and best practices aside, the thing that really hit home for me was the sheer number of places I found that needed fixing. Yes, the code was an issue, but before I even started looking at the code (and thru the judicious application of appropriate tools), I found at least 3 places that affected the server itself.

So the difference between server performance and application performance can be summarized pretty much like this:

Server performance is fundamental to the performance of every application on the server and includes concepts like request tuning; thread and DB connection pooling (both of which affected the app in question); the performance of the connection to the DB and any other external resources (and the performance of the resource itself); JVM settings for things like garbage collection; basically anything that happens on the server but outside any single application. In the ColdFusion world, most of the time this stuff will be a matter of configuration, not code (unless you’re a GlassFish contributor heh), because ColdFusion takes it upon itself to manage most of these things for you.

Application performance, on the other hand, is trickier… because the application runs on the server, if the server is misconfigured somehow, the application may take the blame for something that’s actually a server tuning issue. Application performance may actually involve a particular datasource that’s performing poorly, meaning that applications can be affected by external resources as well… however it can also mean poor choices in application architecture or bad coding practices. But the fundamental difference remains: You can have one poorly performing application on a well-tuned server, but if your server is poorly tuned you’ll suffer performance penalties in pretty much every application.

As I was working through this issue, I was using a process of benchmarking, analyzing, adjusting and re-benchmarking. Each incremental improvement brought with it a new set of questions and a new set of problems. The benchmarks I was running led me to start looking at server settings first, and, once those were solved I was able to work thru some application issues (including some adjustments to code)… but the key was starting in the right place. Running the benchmarks gave me real numbers to work with, both to judge whether or not I was done as well as for comparison to judge the improvements over the previous run. (Hint: Jmeter gives you awesome stats and a good, stable server load to start getting server performance information from the webserver and JVM on up to your application code itself.)

What’s the main take-away from this experience? Hrm… there are many, but I think the key one is that server and application performance tuning aren’t voodoo or black magic and they need to be approached scientifically (read: methodically and consistently). Working in terms of “user time” (“the perceived time a user spends waiting for responses from a website”), using jmeter to simulate load and watch response times, and making sure that you’re looking at your system realistically (because the webserver, app server and application all contribute) and starting objectively from the same point every time you deal with a performance issue will all work to your favor. Using the right tools to gain measurable and/or trackable information from every level of the application will make you a hero.

Over the course of time we’ll probably be blogging about the tools we use for these things, but underneath the tools is the more important part: The Process. Having a methodology for performance tuning is critical and keeping the difference between the performance of your application server and your application itself will help you make sure you start checking in the right place every time and never miss a step.

Because the best application in the world running on a broken server is nothing more than a broken application… know what I mean?