[jira] [Commented] (TIKA-3710) HTML document detected incorrect as message/rfc822

[Tim Allison (Jira)]

    [ 
https://issues.apache.org/jira/browse/TIKA-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539580#comment-17539580
 ] 

Tim Allison commented on TIKA-3710:
-----------------------------------

This works on the test file:

{noformat}
<magic priority="60">
      <match value="&lt;HTML" type="string" offset="0"/>
      <match value="&lt;html" type="string" offset="0"/>
      <match value="&lt;HEAD" type="string" offset="0"/>
      <match value="&lt;head" type="string" offset="0"/>
      <match value="&lt;BODY" type="string" offset="0"/>
      <match value="&lt;body" type="string" offset="0"/>
      <match value="&lt;TITLE" type="string" offset="0"/>
      <match value="&lt;title" type="string" offset="0"/>
      <match value="&lt;DIV" type="string" offset="0"/>
      <match value="&lt;div" type="string" offset="0"/>
      <match value="&lt;h1" type="string" offset="0"/>
      <match value="&lt;H1" type="string" offset="0"/>
      <match value="&lt;h2" type="string" offset="0"/>
      <match value="&lt;H2" type="string" offset="0"/>
      <match value="&lt;h3" type="string" offset="0"/>
      <match value="&lt;H3" type="string" offset="0"/>
    </magic>
    <!-- The magic priority needs to be lower than that of -->
    <!--  files which contain HTML within them, eg mime emails -->
    <magic priority="40">
      <match value="&lt;!DOCTYPE HTML" type="string" offset="0:64"/>
      <match value="&lt;!DOCTYPE html" type="string" offset="0:64"/>
      <match value="&lt;!doctype HTML" type="string" offset="0:64"/>
      <match value="&lt;!doctype html" type="string" offset="0:64"/>
      <match value="&lt;HEAD" type="string" offset="0:64"/>
      <match value="&lt;head" type="string" offset="0:64"/>
      <match value="&lt;TITLE" type="string" offset="0:64"/>
      <match value="&lt;title" type="string" offset="0:64"/>
      <match value="&lt;HTML" type="string" offset="0:64"/>
      <match value="&lt;html" type="string" offset="0:128"/>
    </magic>
{noformat}

> HTML document detected incorrect as message/rfc822
> --------------------------------------------------
>
>                 Key: TIKA-3710
>                 URL: https://issues.apache.org/jira/browse/TIKA-3710
>             Project: Tika
>          Issue Type: Bug
>          Components: detector
>    Affects Versions: 2.3.0
>            Reporter: Sam Stephens
>            Priority: Major
>         Attachments: html-that-looks-like-rfc822.html
>
>
> I'm detecting content types and extracting text from documents using the 
> AutoDetectParser.
> I've received some documents that are HTML fragments generated from emails. 
> The documents are clearly HTML, not emails, but the AutoDetectParser gives me 
> the MIME type message/rfc822 and no text. I've attached an example.
> It looks like the presence of From:, Sent:, and Subject: at the beginning of 
> lines is why the documents are matching RFC822. However, I believe the 
> presence of HTML before these headers means the document is not valid RFC822.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)