In a tutorial posted to the SitePoint PHP blog Claudio Ribeiro tries to answer the question of "who watches the watchmen" (your application's tests) to ensure they're functioning as expected and are correct. In this new tutorial he introduces the Humbug mutation testing tool and how it can be used to verify your own tests.

Regardless of whether you’re working for a big corporation, a startup, or just for yourself, unit testing is not only helpful, but often indispensable. We use unit tests to test our code, but what happens if our tests are wrong or incomplete? What can we use to test our tests? Who watches the watchmen?

[...] Mutation Testing ( or Mutant Analysis ) is a technique used to create and evaluate the quality of software tests. It consists of modifying the tests in very small ways. Each modified version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. Mutations are bugs in our original code and analysis checks if our tests detect those bugs. In a nutshell, if a test still works after it’s mutated, it’s not a good test.

He starts by helping you get it installed (a quick composer require) and creating a simple "calculator" test to show it in use. He then creates the test for the class with some simple testing methods for the basic calculator functionality. He then configures the Humbug installation (via a JSON config file) and executes it on the current tests, sharing the resulting output. He goes through the results showing how to interpret them and points out places where the tests could be improved.