[ 
https://issues.apache.org/jira/browse/TIKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411675#comment-15411675
 ] 

Tim Allison commented on TIKA-2050:
-----------------------------------

Thank you for opening this.  It would be helpful if you summarized the 
problems, submitted unit tests and/or patches.  I'm not sure the regex is at 
fault, but please let me know if it is.

>From what I see:
# charset is defined after 21,000 characters (well beyond our current buffer of 
8192)
# two different charsets are defined (again, well beyond our current buffer).  
By default, we pick the first IIRC.
{noformat}
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Content-Type" content="text/html; charset=gb2312" />
{noformat}

Not sure there's a good way of fixing the second, but we could increase the 
length of the buffer.

Do you see any other problems besides our buffer length?

> HTMLEncodingDetector class fails on some HTML documents
> -------------------------------------------------------
>
>                 Key: TIKA-2050
>                 URL: https://issues.apache.org/jira/browse/TIKA-2050
>             Project: Tika
>          Issue Type: Bug
>          Components: parser
>            Reporter: Shabanali Faghani
>            Priority: Minor
>         Attachments: false-negative-responce-from-HTMLEncodingDetector.zip
>
>
> When [[email protected]] and I were working on 
> [TIKA-2038|https://issues.apache.org/jira/browse/TIKA-2038] I found out that 
> HTMLEncodingDetector class cannot extract charsets from some HTML documents. 
> I’ve attached the HTML documents that HTMLEncodingDetector fails on them. It 
> seems that its regex should be corrected to cover these cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to