On the Ibuildings blog, Cal Evans has written up a post about benchmarking web-based applications and how most of the results could amount to nothing more than useless figures.

The fact of the matter is benchmarking today's web based computer software is almost impossible. In past years, the software was easier to isolate and the number of variables could be controlled so that a useful metric could be produced for buyers to be able to compare two similar products. [...] In any modern web application web servers, database servers, third party APIs and the network all have to be factored in when benchmarking an application. Any one of these not functioning correctly could cause your benchmarks to report false data.

He does mention one of the useful things benchmarks can provide - comparing previous versions of the code against itself.