On Sat, 23 Aug 2014, Marko Rauhamaa wrote:
Isaac Morland <ijmor...@uwaterloo.ca>:
HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-16">
For HTML it's not quite so bad. According to the HTML 4 standard:
[...]
The Content-Type header takes precedence over a <meta> element. I
thought I read once that the reason was to allow proxy servers to
transcode documents but I don't have a cite for that. Also, the <meta>
element "must only be used when the character encoding is organized
such that ASCII-valued bytes stand for ASCII characters" so the
initial UTF-16 example wouldn't be conformant in HTML.
That's not how I read it:
The META declaration must only be used when the character encoding is
organized such that ASCII characters stand for themselves (at least
until the META element is parsed). META declarations should appear as
early as possible in the HEAD element.
<URL: http://www.w3.org/TR/1998/REC-html40-19980424/charset.ht
ml#doc-char-set>
IOW, you must obey the HTTP character encoding until you have parsed a
conflicting META content-type declaration.
From the same document:
--------------------------------------------------------------------------
To sum up, conforming user agents must observe the following priorities
when determining a document's character encoding (from highest priority to
lowest):
An HTTP "charset" parameter in a "Content-Type" field.
A META declaration with "http-equiv" set to "Content-Type" and a value
set for "charset".
The charset attribute set on an element that designates an external
resource.
--------------------------------------------------------------------------
(In the original they are numbered)
This is a priority list - if the Content-Type header gives a charset, it
takes precedence, and all other sources for the encoding are ignored. The
"charset=" on an <img> or similar is only used if it is the only source
for the encoding.
The "at least until the META element is parsed" bit allows for the use of
encodings which make use of shifting. So maybe they start out
ASCII-compatible, but after a particular shift byte is seen those bytes
now stand for Japanese Kanji characters until another shift byte is seen.
This is allowed by the specification, as long as none of the
non-ASCII-compatible stuff is seen before the META element.
The author of the standard keeps a straight face and continues:
I like your way of putting this - "straight face" indeed. The third
option really is a hack to allow working around nonsensical situations
(and even the META tag is pretty questionable). All this complexity
because people can't be bothered to do things properly.
For cases where neither the HTTP protocol nor the META element
provides information about the character encoding of a document, HTML
also provides the charset attribute on several elements. By combining
these mechanisms, an author can greatly improve the chances that,
when the user retrieves a resource, the user agent will recognize the
character encoding.
Isaac Morland CSCF Web Guru
DC 2554C, x36650 WWW Software Specialist
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
