Posts

Showing posts with the label Character Encoding

C/C++ Why To Use Unsigned Char For Binary Data?

Answer : In C the unsigned char data type is the only data type that has all the following three properties simultaneously it has no padding bits, that it where all storage bits contribute to the value of the data no bitwise operation starting from a value of that type, when converted back into that type, can produce overflow, trap representations or undefined behavior it may alias other data types without violating the "aliasing rules", that is that access to the same data through a pointer that is typed differently will be guaranteed to see all modifications if these are the properties of a "binary" data type you are looking for, you definitively should use unsigned char . For the second property we need a type that is unsigned . For these all conversion are defined with modulo arihmetic, here modulo UCHAR_MAX+1 , 256 in most 99% of the architectures. All conversion of wider values to unsigned char thereby just corresponds to truncation to the leas...

Convert Latin1 Characters On A UTF8 Table Into UTF8

Answer : From what you describe, it seems you have UTF-8 data that was originally stored as Latin-1 and then not converted correctly to UTF-8. The data is recoverable; you'll need a MySQL function like convert(cast(convert(name using latin1) as binary) using utf8) It's possible that you may need to omit the inner conversion, depending on how the data was altered during the encoding conversion. After i searched about an hour or two for this answer. I needed to migrate a old tt_news db from typo into a new typo3 version. I already tried convert the charset in the export file and import it back, but didn't get it working. Then i tried the answer above from ABS and startet a update on the table: UPDATE tt_news SET title=convert(cast(convert(title using latin1) as binary) using utf8), short=convert(cast(convert(short using latin1) as binary) using utf8), bodytext=convert(cast(convert(bodytext using latin1) as binary) using utf8) WHERE 1 You can also convert im...