On DZone.com today there's an update about recent additions to Paratest, the parallel PHPUnit test runner (created by Brian Scaturro). He talks some about the benefits of running tests in parallel and shares some of the recent contributions to the project from other developers.

I've already written about my experiments with Paratest. Paratest is a PHPUnit wrapper that allows you to run tests written for PHPUnit in parallel, making us of multiple processes running on the same machine. In a world where cycle time is an important metric, trading resources to get the test suite to finish earlier is a net gain; especially when you're stepping on unstable stones and run the suite very often.

He (Giorgio Sironi) has contributed a new test runner to the project - the "WrapperRunner" that limits the number of processes spawned by the parallel testing tool. Another contribution came from Dimitris Baltas involving the addition of a TEST_TOKEN variable that can be used to uniquely identify each process as they're executing.

On DZone.com Giorgio Sironi has written up a new tutorial showing how to use parallelism with PHPUnit to execute multiple sets of tests at once, hoping for a performance gain.

Of course the cost of coordinating different processes is always going to be present, so we will never reach the theoretical speedup. I'll report later in this article some simulations. The most important constraints come from the design of our test suites. I can only think of two categories of tests as easily parallelizable: unit tests and Selenium tests.

He mentions one specific issue to watch out for - race conditions between the test sets (using the same backend resources). To help solve the issue, he recommends looking into Paratest, a tool that sits on top of PHPUnit and handles the execution of the tests in parallel. He creates some sample tests (they just compute values) and compares the runs of them in single- and multiple-process modes. The difference is a twenty-five percent drop in execution time for the parallel test runs.

In a recent post to his blog David Müller has taken a look at parallel processing in PHP using a few different methods - system calls, fork, and curl.

Since PHP does not offer native threads, we have to get creative to do parallel processing. I will introduce 3 fundamentally different concepts to emulate multithreading as good as possible.

For each of the technologies mentioned above, he provides a simple bit of sample code that does simple tasks like echoing out strings and writing to files. He also includes some benchmarks (take them with a grain of salt) of the three different methods showing how many iterations they could run through in ten seconds. He includes the benchmarking script if you'd like to try it out yourself.

On his blog today Matthew Weier O'Phinney talks about how to run mod_php and FastCGI side-by-side on a Zend Server instance.

I installed Zend Server some time ago, so I'm still on a PHP 5.2 mod_php binary. I have several PHP 5.3 binaries compiled and installed locally for running unit tests and sample scripts already -- so the question was how to keep my 5.2 mod_php running while simultaneously allowing the ability to run selected vhosts in 5.3? The answer can be summed up in one acronym: FastCGI.

He shows how to enable FastCGI in Apache (on Ubuntu), make a virtual host for your site and create a "cgi-bin" directory to contain the script(s) for your PHP versions as CGIs.

New on his blog today Sameer Borate has a post looking at his method for making parallel connections with curl based on this library from Pete Wardens.

Most people use the 'easy' mode - in this mode when we issue multiple requests, the second request will not start until the first one is complete. This is known as synchronous execution, and this is the one we normally use. [...] In [multi] mode all requests can be handled in parallel or asynchronously. And it can be quite handy and time saving on many occasions.

He gives some code examples of how to use the library to simplify the curl connections and requests and pass the result off to a callback when it's done. His more practical example shows how to search for a set of terms on Google and return the results for output.

Kore Nordmann has put together some scripts that will let you create a native parallel job queue using only PHP (and the PCTNL extension).

To make use of multiple cores for some rather long processing operations I needed a way to fork multiple workers from a single PHP script multiple times lately. So I created a small project on github which implements this in a way, so that it should reusable by anybody. This is far from being rocket science, but still might be useful to someone out there.

He uses the ShellJobProvider (extended from JobProvider) to handle the creation of the tasks. His simple example just spawns off a few echo statements and pushes the results to some files. You can get these libraries from hos github account. You'll need PHP 5.3 and the PCTNL extension installed to get it working.

On the Ibuildings blog today there's a new post from Martin Roest looking at parallel processing in PHP scripts and how it can help you with performance and simplifying your applications.

The idea of parallel processing is when you take an atomic transaction or operation called a 'process' and run a couple of those at the same time. [...] In this example I had a PHP CLI script. The purpose of this script was to process remote documents and save it local. It fetched the document from a remote location, processed it and saved the result local - let's call this the transaction. Transactions were done sequentially. It took about 1 second for a transaction to complete and the script had to do roughly 3500 transactions.

Obviously, this script/testing method was not very effective, so he went searching for an alternative. The best option came in the shape of forking processes - spawning off a separate script (via PHP's forking functionality) to do the work on one or multiple entries. It uses the pcntl_fork and pcntl_waitpid functions to spawn and manage these children. Once they're all finished - working in parallel - they return back to the main script to wait for the slower ones to catch up.

John Lim has two new posts covering parallel processing in PHP and how to use this "divide and conquer" idea to not only speed up your code but to make it more maintainable down the road.

In the first post:

One problem we were having is that some of our batch processing jobs were taking too long to run. In order to speed the processing, we tried to split the processing file into half, and let a separate PHP process run each job. [...] Here is our technique for running multiple parallel jobs in PHP. In this example, we have two job files: j1.php and j2.php we want to run.

The code is included for the job files and the "controller" that manages them. In the second article, he builds on this and shows a more practical example - finding the median of a set of records out of a database.

John Lim has a new blog post today sharing his method for simulating parallel processing inside of a PHP application.

One problem we were having is that some of our batch processing jobs were taking too long to run. In order to speed the processing, we tried to split the processing file into half, and let a separate PHP process run each job. Given that we were using a dual core server, each process would be able to run close to full speed (subject to I/O constraints).

He shows the two "jobs" files that just echo out the job name and the number of seconds it's been running and the "control.php" that makes use of streams (pointed at localhost) to start the jobs apart from the main script. Another function checks the stream resource to see if it gets an EOF from it and returns back the output.

The Developer Tutorials blog has posted a tutorial about scraping other website information in parallel (with their permission, of course) with the help of the cURL extension.

For anyone who's ever tried to fetch multiple resources over HTTP in PHP, the logic is trivial, but one key challenge is ever-present: latency delays. While web servers have perfectly good downstream links, latencies can increase script execution time tenfold just by downloading a few external URLs. But there's a simple solution: parallel cURL operations. In this tutorial, I'll show you how to use the "multi" functions in PHP's cURL library to get around this quickly and easily.

He starts with a basic cURL example, grabbing the content from example.com and putting it into a variable. He modifies this to make it a bit more complex and to run multiple fetches in parallel - creating more than one cURL object and using the culr_multi_* methods to manage them.