Hi,
After having a look at HTML5 editor's draft I believe that 8.2.2.3
incorrectly instructs changing UTF-16 to UTF-8.
UTF-16 without BOM cannot be detected using the sniffing algorithm
because is incompatible with ASCII. But the browser may guess (step 6.)
that it's UTF-16 but it will only be tentative.
After the parser may find and encoding specified in the UTF-16 text.
If the encoding found is UTF-16 then that is instructed by step 1. of
8.2.2.3 to be changed to UTF-8 that is definitely wrong.
Another problem is that if you were able to find an encoding name other
than UTF-16 in a valid HTML code decoded as if it were UTF-16 you
shouldn't restart parsing because if it isn't UTF-16 then the encoding
found is not accurate either.
4.2.5.5 also states:
If an HTML document does not start with a BOM, and if its encoding is
not explicitly given by Content-Type metadata, then the character
encoding used must be an ASCII-compatible character encoding
and
If an HTML document contains a meta element with a charset attribute or
a meta element in the Encoding declaration state, then the character
encoding used must be an ASCII-compatible character encoding.
These together are equivalent to saying that UTF-16 without BOM is not
allowed but I believe that this was not the intent. If it really is I
would prefer to have an explicit note about this.
Encoding found by parsing using UTF-16 should be UTF-16 and any other
values should be treated as a parse error.
Permitting UTF-16 without BOM makes sense because encoding autodetection
is permitted as well and ASCII compatible encodings having encoding
specified using <meta> will not reach the autodetection stage.
Best regards,
Kornél Pál
