In a new post to his blog Danne Lundqvist looks at a common pitfall that could trip you up if you're not careful with your UTF-8 data - not looking for the UTF byte order mark that tells the application if it needs to be handled as UTF content.

One such thing is the occurrence of the UTF byte order mark, or BOM. [...] For UTF-8, especially on Windows, it has become more and more common to use it to indicate that the file is indeed UTF. Most text editors handle this well and you won’t ever see these bytes. As it should be.

He points out what could cause an issue - using strcmp or substr but it can be prevented by looking at and removing those first three bytes if needed. He includes a snippet of code that does just that.

Sean Coates has reposted an article that was originally published in php|architect magazine covering UTF-8 and proper Unicode encoding.

If I had to guess, I would estimate that I've spent somewhere in the range of 40 hours wrangling UTF-8 in the past 3 months, which is not only expensive for my employer, but also disheartening as a developer who's got real work to do. Admittedly, this number is inflated, due to the heavy development cycle we completed with the launch of our new site.

Sean goes on to talk about Unicode issues in general (partially supported in some places, too many points of failure) and some of his other experiences with "the UTF-8 monster" that have given him trouble over time.