Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

James Cohen's Blog:
Poor Man’s Parallelization for Batch Processing Jobs
May 18, 2011 @ 11:56:31

James Cohen has a quick post about what he calls a "poor man's parallelization" for working with batch jobs. It takes in parameters that tell it which set of jobs to run through when looping.

One common problem that I’ve seen time and time again with batch processing jobs (generally cronjobs) is that when they’re written they run quickly. Over time their workload grows until eventually it’s unacceptably slow. [...] To create a simple of way of separating the jobs in a consistent way we can use the modulus operator. It just calculates the remainder of two numbers. It’s a common arithmetic operator in almost all languages so this technique is pretty portable.

His proof-of-concept script takes in two parameters, the starting job number and the number to increment. His example is user IDs, but this type of script could be used for anything with an ID number. The script is then run from the command line with the parameters of your choosing.

tagged: batch processing separation modulus

Link:

Zoe Slattery's Blog:
Oh no! Not more tests!
Jul 09, 2009 @ 09:42:16

As Zoe Slattery mentions in her new post more testing for PHP is a good thing, but has come with a bit of a problem.

One of the problems with the incredible level of success of the PHP TestFest is that PHP gets more tests, 887 more to be precise. Well, isn't that the point? Yes, but it's beginning to take *forever* to run them which is a bad thing and as TestFests get bigger and better this will only get worse.

As a result, more work has been put into a project started a while back - a tool to allow more than one test to run at once (instead of linearly). This grab shows some of the resulting gains in speed. It jumps to about half to the time to make it through the ~7700 tests.

tagged: batch run performance unittest

Link:

Symfony Blog:
Batches are dead, long life to tasks!
Jun 16, 2008 @ 12:05:36

On the Symfony blog today, Romain Dorgueil shows how to create tasks to help automate things in your application like database updates, console scripts or other "repetitive maintenance tasks".

Symfony 1.1 extends symfony 1.0 pake tasks to create a powerful and uniform command line utility for your projects, fully integrated with the symfony Command Line Interface (CLI).

This means that is has the abilities to automatically support a "help" parameter, to grab a current task list the app is using, correct handling of the input parameters, set up a good environment and make sure that the source is readable. The post shows how to create a new task for your application - their examples are a "doNothingTask" that, well, does a lot of nothing and the typical "doHelloWorldTask" that just echoes.

They show how to define the task (including a namespace for it) and how to run it, outputing the results of the execute() method ("I did nothing successfully!" in the first case and a string of passed in parameters in the second.

tagged: task batch tutorial symfony framework helloworld

Link:

IBM developerWorks:
Batch processing in PHP
Dec 07, 2006 @ 09:06:00

Both this post on the Zend Developer Zone and tis post on the International PHP Magazine's website point to a new article over on the IBM developerWorks website by Jack Herrington, Batch processing with PHP.

What do you do when you have a feature in your Web application that takes longer than a second or two to finish? You need some type of offline processing solution. Check out several methods for offline servicing of long-running jobs in your PHP application.

He talks about cron and its role in offline processing (including a basic primer on its format) before getting into the example itself. He looks at three examples:

  • building an email queue
  • building a generic queue system
  • dumping out the database
Each example comes complete with code and descriptions to help you work them up on you very own system.

tagged: batch process cron mail quene generic database dump batch process cron mail quene generic database dump

Link:

IBM developerWorks:
Batch processing in PHP
Dec 07, 2006 @ 09:06:00

Both this post on the Zend Developer Zone and tis post on the International PHP Magazine's website point to a new article over on the IBM developerWorks website by Jack Herrington, Batch processing with PHP.

What do you do when you have a feature in your Web application that takes longer than a second or two to finish? You need some type of offline processing solution. Check out several methods for offline servicing of long-running jobs in your PHP application.

He talks about cron and its role in offline processing (including a basic primer on its format) before getting into the example itself. He looks at three examples:

  • building an email queue
  • building a generic queue system
  • dumping out the database
Each example comes complete with code and descriptions to help you work them up on you very own system.

tagged: batch process cron mail quene generic database dump batch process cron mail quene generic database dump

Link:

Wez Furlong's Blog:
Background/batch/workflow processing with PDO::PGSQL
Oct 27, 2006 @ 08:42:00

Wez Furlong wants a bit more out of his PHP script, naemly the ability to be able to process things in the background without tying up or immediately using the script that's running.

In my recent talk on sending mail from php I mention that you want to avoid sending mail directly from a web page. A couple of people have asked me how to implement that, and one of the suggestions I have is to queue your mail in a database table and have some other process act on that table.

He gives a solution that's a bit more optimized for this solution than just polling the same information over and over - using PDO and LISTEN/NOTIFY processing along with transactions to make the CLI script only grab information when there's something new.

tagged: pdo pgsql postgresql mail cli background batch processing pdo pgsql postgresql mail cli background batch processing

Link:

Wez Furlong's Blog:
Background/batch/workflow processing with PDO::PGSQL
Oct 27, 2006 @ 08:42:00

Wez Furlong wants a bit more out of his PHP script, naemly the ability to be able to process things in the background without tying up or immediately using the script that's running.

In my recent talk on sending mail from php I mention that you want to avoid sending mail directly from a web page. A couple of people have asked me how to implement that, and one of the suggestions I have is to queue your mail in a database table and have some other process act on that table.

He gives a solution that's a bit more optimized for this solution than just polling the same information over and over - using PDO and LISTEN/NOTIFY processing along with transactions to make the CLI script only grab information when there's something new.

tagged: pdo pgsql postgresql mail cli background batch processing pdo pgsql postgresql mail cli background batch processing

Link: