[
https://issues.apache.org/jira/browse/TIKA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106893#comment-13106893
]
Ken Krugler commented on TIKA-431:
----------------------------------
Some other things I should have mentioned with regards to this patch:
* Anybody who was returning the charset via CONTENT_ENCODING now returns it via
CONTENT_TYPE, in the ...; charset=xxx parameter.
* If the charset is specified via Metadata.CONTENT_TYPE, and it's valid
(supported, etc) then TXTParser will no longer use ICU4J's charset detector
code. This is to match the behavior of the HtmlParser code, and the essence of
TIKA-539. We could revisit this, if people feel that depending on things like
server response headers is too risky.
* The charset detection algorithm in HtmlParser matches what was
proposed/discussed in TIKA-539. Namely the <meta> tag or incoming metadata
content-type charset is used if it's valid, and only exists in one or the
other, or is the same in both. Otherwise ICU4J is used.
> Tika currently misuses the HTTP Content-Encoding header, and does not seem to
> use the charset part of the Content-Type header properly.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TIKA-431
> URL: https://issues.apache.org/jira/browse/TIKA-431
> Project: Tika
> Issue Type: Bug
> Components: general
> Reporter: Erik Hetzner
> Assignee: Ken Krugler
> Attachments: TIKA-431.patch
>
>
> Tika currently misuses the HTTP Content-Encoding header, and does not seem to
> use the charset part of the Content-Type header properly.
> Content-Encoding is not for the charset. It is for values like gzip, deflate,
> compress, or identity.
> Charset is passed in with the Content-Type. For instance: text/html;
> charset=iso-8859-1
> Tika should, in my opinion, do the following:
> 1. Stop using Content-Encoding, unless it wants me to be able to pass in
> gzipped content in an input stream.
> 2. Parse and understand charset=... declarations if passed in the Metadata
> object
> 3. Return charset=... declarations in the Metadata object if a charset is
> detected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
