[
https://issues.apache.org/jira/browse/TIKA-4370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17921798#comment-17921798
]
Tim Allison commented on TIKA-4370:
-----------------------------------
Hi [~subbudvk], this is long standing problem. Figuring out when a file is a
binary vs a non-ascii text file when there is no filename hint is not trivial.
Depending on your application, you could configure the text parser to parse
{{application/octet-stream}} so that you get content out. Another option would
be to integrate the tika-eval module in a new detector that runs encoding
detection on application/octet stream, extracts the text and then looks at the
text statistics (language id, out of vocabulary etc) and determines if there's
enough "non-garbage" to count as a text file.
> SJIS Encoded Files Can't be Detected
> ------------------------------------
>
> Key: TIKA-4370
> URL: https://issues.apache.org/jira/browse/TIKA-4370
> Project: Tika
> Issue Type: Bug
> Reporter: Subbu
> Priority: Major
>
> When character encoding of file is SJIS, without file name in the metadata,
> most files content-type detected as application/octet-stream. Is there zero
> support for SJIS?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
