[jira] [Comment Edited] (TIKA-2471) Tab-prefixed message body lines in Mbox interpreted as headers
[ https://issues.apache.org/jira/browse/TIKA-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208769#comment-16208769 ] Luis Filipe Nassif edited comment on TIKA-2471 at 10/18/17 3:47 AM: Hi Matthew, If I remember correctly, some headers were not being extracted by the RFC822PARSER after the refactoring of Mboxparser, so that logic was added to get the missed headers back, right [~thaichat04]? I think it may be better to fix the RFC822PARSER instead. The windows-1252 charset was used initially to facilitate locating newlines and the "From" delimiter, I think it does not corrupt content because the chars are converted back to bytes and there is one to one mapping between chars and bytes with this charset. But I think that shouldn't be added to contentType metadata of mbox container. was (Author: lfcnassif): Hi Matthew, If I remember correctly, some headers were not being extracted by the RFC822PARSER after the refactoring of Mboxparser, so that logic was added to get the missed headers back, right [~thaichat04]? I think it may be better to fix the RFC822PARSER instead. The windows-1252 charset was used initially to facilitate locating newlines and the "From" delimiter. I think that shouldn't be added to contentType metadata of mbox container. > Tab-prefixed message body lines in Mbox interpreted as headers > -- > > Key: TIKA-2471 > URL: https://issues.apache.org/jira/browse/TIKA-2471 > Project: Tika > Issue Type: Bug > Components: parser >Affects Versions: 1.16 >Reporter: Matthew Caruana Galizia > Labels: message, rfc822 > Attachments: mbox > > > The mbox parser code is overly optimistic. It parses the entire message > looking for anything that matches a header pattern, wherever it occurs in a > line! > It looks to me like the parsing logic is in desperate need of a refactor. But > more to the point, what is the idea behind setting the headers in the > MboxParser if they're going to be set by the RFC822Parser in any case? > Also, out of curiosity, why does the parser force Windows-1252 as the charset? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (TIKA-2471) Tab-prefixed message body lines in Mbox interpreted as headers
[ https://issues.apache.org/jira/browse/TIKA-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206418#comment-16206418 ] Tim Allison edited comment on TIKA-2471 at 10/16/17 7:30 PM: - That looks totally hosed. Thank you for opening this and supplying an example triggering file. bq. But more to the point, what is the idea behind setting the headers in the MboxParser if they're going to be set by the RFC822Parser in any case? TIKA-1244 brought that behavior in. Before that, emails weren't treated as embedded files if I understand correctly. bq. why does the parser force Windows-1252 as the charset? Again, no idea, -but I suspect that was because of the rfc822 method of encoding-. I simply have no idea. Are you able to share an example where this corrupts the content? was (Author: talli...@mitre.org): That looks totally hosed. Thank you for opening this and supplying an example triggering file. bq. But more to the point, what is the idea behind setting the headers in the MboxParser if they're going to be set by the RFC822Parser in any case? TIKA-1244 brought that behavior in. Before that, emails weren't treated as embedded files if I understand correctly. bq. why does the parser force Windows-1252 as the charset? Again, no idea, but I suspect that was because of the rfc822 method of encoding. Are you able to share an example where this corrupts the content? > Tab-prefixed message body lines in Mbox interpreted as headers > -- > > Key: TIKA-2471 > URL: https://issues.apache.org/jira/browse/TIKA-2471 > Project: Tika > Issue Type: Bug > Components: parser >Affects Versions: 1.16 >Reporter: Matthew Caruana Galizia > Labels: message, rfc822 > Attachments: mbox > > > The mbox parser code is overly optimistic. It parses the entire message > looking for anything that matches a header pattern, wherever it occurs in a > line! > It looks to me like the parsing logic is in desperate need of a refactor. But > more to the point, what is the idea behind setting the headers in the > MboxParser if they're going to be set by the RFC822Parser in any case? > Also, out of curiosity, why does the parser force Windows-1252 as the charset? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
