In this recent post to the SitePoint PHP blog Bruno Skvorc looks at using a popular tool from the Apache project, Apache Bench (or just "ab") to stress-test your application.

There’s no telling when your app might attract a throng of visitors at once. [...] Regardless of the reason, massive influxes of visitors are a double-edged sword: they get you what you always wanted – a chance to prove your worth to a large chunk of the internet’s population – but also often bring with them what you always feared: absolute downtime. [Some] platforms usually offer plugins that can optimize your application while it’s up, so you can fine tune it as you go along, but why not try and predict issues while still developing locally and save yourself time, money and effort in the long run?

He bases the testing off of a Laravel Homestead virtual machine instance and tests a simple "hello world" PHP page to minimize any overhead from other processing. He includes the commands to make a simple ab request and mentions the kinds of request it provides on completion. He moves on from there to something a bit more complex - an actual Laravel-based application using the default "HomeController" and "showWelcome" action/view combination.

On his site today Lukasz Kujawa has a post that compares some performance benchmarks of several popular PHP frameworks including Slim, CodeIgniter, Laravel, Symfony2 and Zend Framework 2.

There are many assumptions around performance of different PHP frameworks. I frequently hear strong opinions about superiority X over Y in this context. There are companies writing new PHP frameworks from scratch because available solutions are too slow for them. What does it really mean? Does the framework performance matters? Before answering this questions lets check how slow is your framework!

He took the "quick start" projects provided for each of the examples and ran some tests with the Apache Benchmark (ab) tool against EC2 instances, all configured the same way. The results weren't overly surprising with Slim beating the others hands down (it's a micro-framework after all) and Kohana and CodeIgniter coming in second and third. The frameworks with more overhead like Zend Framework and Symfony ranked some of the slowest.

Robert Basic has a recent post showing you how to use the "cookie jar" functionality included with Apache's "ab" benchmarking tool to get behind your PHP-based login with a simple curl and grep combo.

Tonight I decided to relax a bit and what better way of relaxing is there for a geek then to do some bash scripting?! So for fun and no profit I decided to try and benchmark pages with ab, Apache HTTP server benchmarking tool, which are behind a login. Turns out, it’s pretty easy after reading some man pages.

He includes an example of the format of the "cookie jar" and the shell script he used to grab the PHP session ID from it and inject it into the "ab" call. The script is on github.

Shay Ben Moshe has a new post to his blog today looking at some of the benefits that using the persistent connections offered in PDO can have on your application.

PDO is an abstraction layer for database connections in PHP, and it became increasingly popular in the past few years. PDO gives us the option to use a persistent connection. If we don't use this option, a new connection is created for each request. If we do use this option, the connection is not closed at the end of the script, and it is then re-used by other script requests.

He shares the setup for his testing (hardware and MySQL configuration) and some of the results from his tests using the Apache ab tool for making web requests against an application. You'll need to download the archived file to see the results, though. It also includes the files he used to test with, comparing regular connections to the persistent ones.

Paul Jones has come back around and revisited the benchmarking setup he's created and has rerun some of the baselines on a new, clean EC2 instance and posted the results to his blog. These benchmarks were run using Apache's ab, Acme's http_load and joedog's siege.

I thought it might be interesting to see what each of them reports for the baseline "index.html" and "index.php" cases on the new Amazon EC2 setup (using a 64-bit OS on an m1.large instance). The results follow (all are at 10 concurrent users, averaged over 5 one-minute runs).

In his results, he shows quite different numbers for the "absolute" requests per second that each of the tools registered. There were some differences from the previous benchmark runs that could have been caused by the update in Ubuntu version and using a 64-bit instance over a 32-bit instance on the EC2 setup.

Vidyut Luther has been looking into testing his application, not with unit tests or performance testing, but with multivariate tests (A/B testing).

I was curious if anyone has any experience with Multivariate (A/B) testing with PHP Frameworks. I found http://phpabtest.com/ yesterday, but I haven’t heard of anyone using it in production yet. Nor have I been able to gather how easily or well it integrates with Symfony, or other frameworks that introduce some level of caching.

He lays out what he's looking to measure, but isn't sure the tool is up to the job. He's open for suggestions as to what the community at large uses for this sort of thing. Besides the one he mentioned, there's a few other PHP-powered ones (some listed here) but nothing that stands out from the crowd.

In a recent post to The Bakery (the CakePHP resource) Jonathan Hendler has created a simple A/B test helper using the CakePHP framework and jQuery.

Knowing what is driving the user experience is key to the success of an application. Subtle changes in the interface can cause dramatic shifts in user behavior. Here, A/B tests display two (or more) language choices or color choices to a user (any HTML). Metrics are measured in two ways - 1) did the user click on the button at all and 2) how long did it take them to find it from the moment the page has started loading?

The helper tracks the user through the site and adds the results to a testing table (abtests). The code is included - model, component, controller, helper class and the jquery code to make it all work together. A few usage examples are also thrown in near the end.

Paul Jones, who is obsessed (is that the right word?) with keeping benchamrks on recent versions of several popular PHP frameworks, has posted another look at a slight change in his testing method - a move away from the Apache ab tool and towards seige (a product of JoeDog Software).

It turns out that the siege tool from JoeDog Software is more accurate in reporting static HTML and PHP responsiveness. This is confirmed through Paul Reinheimer as well, who reported the expected responsiveness on other systems.

The over-reporting from ab means that all my previous reporting on benchmarks is skewed too low when comparing framework responsiveness to PHP’s maximum responsiveness. As such, I have re-run all the previously published benchmarks using siege instead of ab.

Included in this new post are the "more correct" numbers as produced by siege. His baselines turned out a bit more rational and the statistics of the frameworks seemed to jump a bit because of it, but the percentages in comparing between the frameworks is still about the same.

While running some benchmarks on some of the more popular PHP frameworks, Paul Jones came across something interesting - a PHP request was handled faster than a static HTML request.

With the help of the great guys at Slicehost.com, I am attempting to run my benchmark series on a virtual private server, to compare with EC2. However, I’m seeing a very strange result for the baselines: a PHP page delivers more requests-per-second than a static HTML page.

According to his results, his runs of ab (the Apache benchmarking tool) showed that requests against the index.html file (versus the index.php) came in at almost 400 requests per second slower. As it turns out, though, the problem of the skewed results wasn't in the serving up of the requests but in the ab tool itself. He picked up another testing tool, seige, and ran that with the same requests and got more correct results.

On the PHP::Impact blog there's a recent post that details a statistic Paul Jones found with the Zend Framework's performance - a drop in the response time of the front controller between the 1.0 and 1.5 releases.

The most important factor in making a Web application fast is its basic design. You must also know what kinds of processing your framework is doing, and what its bottlenecks are. The best way to find the performance bottlenecks is to monitor the performance counters and to have a thorough understanding of the framework your application is using.

Federico runs some benchmarks of his own (using the Apache benchmarking tool, ab) both with a default "base app" and with a more optimized "take out what you don't need" version.