On Reddit.com there's a recent post asking about testing developers as a part of the hiring process and how difficult it should be.

I developed an exercise for applicants. They must use an open source framework to make a simple bulletin board. It requires account creation, login, thread creation, message creation, unread message indication per thread, and logout. Every applicant is given this exercise after passing an initial test. The initial test is a simple disqualifier and is not challenging for anyone not lying on their resume.

There's some great discussion in the comments about things like the suggested time limit, opinions on other methods that could yield the same results, thoughts on the validity of even taking the test based on the level of the job. The popular opinion, though, is that it's too much work for an applicant to go through unless there's a very specific need. As one commentor put it:

Most of the excellent programmers will have jobs. Those that don't will probably have several options (unless you're in a very small market) for where to work. When it comes down to applying to the place that wants you to spend a bunch of your own time on their application, or some other company... They'll go with some other company.