"’" Showing On Page Instead Of " ' "


Answer :

So what's the problem,

It's a ’ (RIGHT SINGLE QUOTATION MARK - U+2019) character which is being decoded as CP-1252 instead of UTF-8. If you check the encodings table, then you see that this character is in UTF-8 composed of bytes 0xE2, 0x80 and 0x99. If you check the CP-1252 code page layout, then you'll see that each of those bytes stand for the individual characters â, € and ™.

and how can I fix it?

Use UTF-8 instead of CP-1252 to read, write, store, and display the characters.

I have the Content-Type set to UTF-8 in both my <head> tag and my HTTP headers:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

This only instructs the client which encoding to use to interpret and display the characters. This doesn't instruct your own program which encoding to use to read, write, store, and display the characters in. The exact answer depends on the server side platform / database / programming language used. Do note that the one set in HTTP response header has precedence over the HTML meta tag. The HTML meta tag would only be used when the page is opened from local disk file system instead of from HTTP.

In addition, my browser is set to Unicode (UTF-8):

This only forces the client which encoding to use to interpret and display the characters. But the actual problem is that you're already sending ’ (encoded in UTF-8) to the client instead of ’. The client is correctly displaying ’ using the UTF-8 encoding. If the client was misinstructed to use, for example ISO-8859-1, you would likely have seen ââ¬â¢ instead.

I am using ASP.NET 2.0 with a database.

This is most likely where your problem lies. You need to verify with an independent database tool what the data looks like.

If the ’ character is there, then you aren't connecting to the database correctly. You need to tell the database connector to use UTF-8.

If your database contains ’, then it's your database that's messed up. Most probably the tables aren't configured to use UTF-8. Instead, they use the database's default encoding, which varies depending on the configuration. If this is your issue, then usually just altering the table to use UTF-8 is sufficient. If your database doesn't support that, you'll need to recreate the tables. It is good practice to set the encoding of the table when you create it.

You're most likely using SQL Server, but here is some MySQL code (copied from this article):

CREATE DATABASE db_name CHARACTER SET utf8; CREATE TABLE tbl_name (...) CHARACTER SET utf8;

If your table is however already UTF-8, then you need to take a step back. Who or what put the data there. That's where the problem is. One example would be HTML form submitted values which are incorrectly encoded/decoded.

Here are some more links to learn more about the problem:

  • The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!), from our own Joel.
  • Unicode - How to get the characters right?, with more concise and practical information, solutions are targeted on Java environments.
  • How to setup your PHP site to use UTF8, targeted on PHP environments.

Ensure the browser and editor are using UTF-8 encoding instead of ISO-8859-1/Windows-1252.

Or use &rsquo;.


I have some documents where … was showing as … and ê was showing as ê. This is how it got there (python code):

# Adam edits original file using windows-1252 windows = '\x85\xea'  # that is HORIZONTAL ELLIPSIS, LATIN SMALL LETTER E WITH CIRCUMFLEX  # Beth reads it correctly as windows-1252 and writes it as utf-8 utf8 = windows.decode("windows-1252").encode("utf-8") print(utf8)  # Charlie reads it *incorrectly* as windows-1252 writes a twingled utf-8 version twingled = utf8.decode("windows-1252").encode("utf-8") print(twingled)  # detwingle by reading as utf-8 and writing as windows-1252 (it's really utf-8) detwingled = twingled.decode("utf-8").encode("windows-1252")  assert utf8==detwingled

To fix the problem, I used python code like this:

with open("dirty.html","rb") as f:     dt = f.read() ct = dt.decode("utf8").encode("windows-1252") with open("clean.html","wb") as g:     g.write(ct)

(Because someone had inserted the twingled version into a correct UTF-8 document, I actually had to extract only the twingled part, detwingle it and insert it back in. I used BeautifulSoup for this.)

It is far more likely that you have a Charlie in content creation than that the web server configuration is wrong. You can also force your web browser to twingle the page by selecting windows-1252 encoding for a utf-8 document. Your web browser cannot detwingle the document that Charlie saved.

Note: the same problem can happen with any other single-byte code page (e.g. latin-1) instead of windows-1252.


Comments

Post a Comment

Popular posts from this blog

Converting A String To Int In Groovy

Answer : Use the toInteger() method to convert a String to an Integer , e.g. int value = "99".toInteger() An alternative, which avoids using a deprecated method (see below) is int value = "66" as Integer If you need to check whether the String can be converted before performing the conversion, use String number = "66" if (number.isInteger()) { int value = number as Integer } Deprecation Update In recent versions of Groovy one of the toInteger() methods has been deprecated. The following is taken from org.codehaus.groovy.runtime.StringGroovyMethods in Groovy 2.4.4 /** * Parse a CharSequence into an Integer * * @param self a CharSequence * @return an Integer * @since 1.8.2 */ public static Integer toInteger(CharSequence self) { return Integer.valueOf(self.toString().trim()); } /** * @deprecated Use the CharSequence version * @see #toInteger(CharSequence) */ @Deprecated public static Integer toInteger(String self) { return toInteg
Read more

"Cannot Create Cache Directory /home//.composer/cache/repo/https---packagist.org/, Or Directory Is Not Writable. Proceeding Without Cache"

Answer : if anyone pass through here, this is shorter solution: sudo chown -R $USER $HOME/.composer it seems to me the group information is missing in your command sudo chown -R <user> /home/<user>/.composer/cache/repo/https---packagist.org Shoud be sudo chown -R <user>:<group> /home/<user>/.composer/cache/repo/https---packagist.org But to avoid other permission issues, I would rather advise: sudo chown -R <user>:<group> /home/<user>/.composer/cache (you'll need access to other folders in there) and sudo chown <user>:<group> /home/<user>/.composer To make sure your user has permissions enough on the global composer folder. Mind the missing recursion so the user don't own keys created by root. If you need to find out the group: groups <user>
Read more

Android How Can I Convert A String To A Editable

Answer : As you probably found out, Editable is an interface so you cannot call new Editable() . However an Editable is nothing more than a String with Spannables, so use SpannableStringBuilder: Editable editable = new SpannableStringBuilder("Pass a string here"); If you only want to change the Spannables, never the text itself, you can use a basic SpannableString. Use Editable.Factory.getInstance().newEditable(str) From the android documentation: Returns a new SpannedStringBuilder from the specified CharSequence. You can override this to provide a different kind of Spanned.
Read more