News Feed
Sections




News Archive
feed this:

Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

Ahmed Shreef's Blog:
iconv misunderstands UTF-16 strings with no BOM
August 27, 2010 @ 13:36:56

Ahmed Shreef has a recent post to his blog about an issue he had converting UTF-16 strings over to UTF-8 with the iconv functionality in PHP. Specifically, he ended up with "rubbish unreadable characters" after the conversion.

I had a problem last week with converting UTF-16 encoded strings to UTF-8 using PHP's iconv library on a Linux server. my code worked fine on my machine but the same code resulted in a rubbish unreadable characters on our production server.

In his example (a basic "Hello World" in Arabic) he notes that there's no byte order mark on the string and, because of this, the iconv feature tries to guess if it's big-endian or little-endian. This guessing varies from machine to machine resulting in the inconsistencies he saw. The solution is to define the "to" and "from" for the conversion manually rather than letting it just guess.

0 comments voice your opinion now!
byteordermark bom iconv utf16 utf8 convert


Danne Lundqvist's Blog:
Detecting UTF BOM - byte order mark
April 29, 2010 @ 11:47:03

In a new post to his blog Danne Lundqvist looks at a common pitfall that could trip you up if you're not careful with your UTF-8 data - not looking for the UTF byte order mark that tells the application if it needs to be handled as UTF content.

One such thing is the occurrence of the UTF byte order mark, or BOM. [...] For UTF-8, especially on Windows, it has become more and more common to use it to indicate that the file is indeed UTF. Most text editors handle this well and you won't ever see these bytes. As it should be.

He points out what could cause an issue - using strcmp or substr but it can be prevented by looking at and removing those first three bytes if needed. He includes a snippet of code that does just that.

0 comments voice your opinion now!
byteordermark utf utf8 utf16 detect



Community Events





Don't see your event here?
Let us know!


podcast package tool framework update opinion security release version interview introduction voicesoftheelephpant community application laravel series symfony composer library language

All content copyright, 2014 PHPDeveloper.org :: info@phpdeveloper.org - Powered by the Solar PHP Framework