On his blog, Sebastian Nohn points out a tool that he's created to harness the functionality of CruiseControl to help you keep your PHP updates a bit more sane.

Assuring compatibility with new PHP versions is not always easy: Features are added and changed in minor versions, bugs you didn't even know they exist and having been open for years "suddenly" get fixed.

Assuring compatibility however is easy with a little help of CruiseControl as long as you have automated tests for your software: PHP on Cruise.

His tool grabs the latest stable PHP release, compiles it with defined configure options and runs a set of user-defined tests. This makes it simple to run an automated compile and test of your entire installation to ensure that noting has been broken.