Double encoded UTF-8 strings in C#
This article shows how to convert a string that has been double encoded using UTF-8.
For example, say you have the string MÃ¼ller instead of the string Müller.
How did it happen?
The letter ü is encoded in UTF-8 as 2 bytes: 195 and 188
If you encoded the bytes again then the 195 converts to 195 and 131 which is the Ã
And the 188 converts to 194 and 188 which is the ¼
How can it be converted back to what it should look like?
The following function will convert the double encoded string back to the original value…
private string decodeUTF8String(String utf8Str)
System.Text.Encoding iso = System.Text.Encoding.GetEncoding(“ISO-8859-1”);
System.Text.Encoding utf8 = System.Text.Encoding.UTF8;
byte utfBytes = utf8.GetBytes(utf8Str);
byte isoBytes = System.Text.Encoding.Convert(utf8, iso, utfBytes);
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
How does this relate to barcodes?
Some PDF-417 barcodes may contain data that has already been encoded using UTF-8 and when we read the barcode we encode it again using UTF-8, giving a double encoded string. In the win32 DLL interface the work-around is simply to set the Encoding property to 0, but the above is necessary in the .Net interface.