[
https://issues.apache.org/jira/browse/TIKA-4370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17924066#comment-17924066
]
Subbu commented on TIKA-4370:
-----------------------------
Haha new thing called TextAndCSVParser, :D I was talking about TXTParser and
TextDetector.
Sorry if i am not clear earlier. I see after we disabled file name metadata, it
came here
[https://github.com/apache/tika/blob/be5003c8c55ebcc39ffecfe8e95ac5eef37b803f/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java#L293]
and determined the type as application/octet-stream while it is a shift-jis
text/plain file.
I was asking here
[https://github.com/apache/tika/blob/be5003c8c55ebcc39ffecfe8e95ac5eef37b803f/tika-core/src/main/java/org/apache/tika/detect/TextDetector.java#L131]
in addition to stats.isMostlyAscii() || stats.looksLikeUTF8(), can we do
TXTParser and if type is determined to Shift-JIS we return text/plain.
But that means tika-core is having a dependency on tika-parsers, which I think
isn't correct?
Also, since we disabled file name I am assuming even if it works the other
textual shift-jis file such as csv files we will miss detection here
[https://github.com/apache/tika/blob/be5003c8c55ebcc39ffecfe8e95ac5eef37b803f/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java#L557]
[~tallison]
> SJIS Encoded Files Can't be Detected
> ------------------------------------
>
> Key: TIKA-4370
> URL: https://issues.apache.org/jira/browse/TIKA-4370
> Project: Tika
> Issue Type: Bug
> Reporter: Subbu
> Priority: Major
>
> When character encoding of file is SJIS, without file name in the metadata,
> most files content-type detected as application/octet-stream. Is there zero
> support for SJIS?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
