Gerard Bouchar created TIKA-2671:
------------------------------------
Summary: HtmlEncodingDetector doesnt take provided metadata into
account
Key: TIKA-2671
URL: https://issues.apache.org/jira/browse/TIKA-2671
Project: Tika
Issue Type: Bug
Reporter: Gerard Bouchar
org.apache.tika.parser.html.HtmlEncodingDetector ignores the document's
metadata. So when using it to detect the charset of an HTML document that came
with a conflicting charset specified at the transport layer level, the encoding
specified inside the file is used instead.
This behavior does not conform to what is [specified by the W3C for determining
the character encoding of HTML
pages|https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding].
This causes bugs like NUTCH-2599.
If HtmlEncodingDetector is not meant to take into account meta-information
about the document, then maybe another detector should be provided, that would
be a CompositeDetector including, in that order:
* a new, simple, MetadataEncodingDetector, that would simply return the
encoding
* the existing HtmlEncodingDetector
* a generic detector, like UniversalEncodingDetector
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
