Sat, 04 Oct 2008
What are your most expensive websites to run? Patching Apache to find out
When running a busy webserver, one may want to know how much server time is spent preparing each request. That would be especially useful if broken-down per web site you host. Server processing time indicates things like how long MySQL queries took, or how loaded the disks are; in general, they are the measure of how difficult it was to answer a request. It may also be interesting to compare server time spent processing a request today to the same request's time in the past as an indication of how system changes (upgraded disks, more complex filesystem) have affected your ability to process web requests.
Apache's mod_log_config lets you log how long a request takes from start to end, which includes the amount of time taken to send the actual data. That can be imagined as server_processing_time + time_to_send_data_to_client. I wasn't interested in seeing how slow or fast clients' net connections were.
In a project I named vhost_effort, I wrote a patch to Apache to be able to log just that server time spent from the start of the request to when the request is ready to be sent. That work was done at Creative Commons, and the software results are available under the Apache 2.0 license. vhost_effort.py is a hack that generates a pie graph for how much server time is spent on each vhost (among other sorts of visualizable statistics). I began thinking of using a visualizer for disk usage to make the pie graph interactive, but by the time I was nearly done working that out we had already gathered all the data we needed.
My projects page has a link to the code in the Creative Commons Subversion repository. I did write about this at labs.creativecommons.org a year ago also.
