[
https://issues.apache.org/jira/browse/TIKA-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780855#comment-16780855
]
Tim Allison commented on TIKA-2833:
-----------------------------------
One of the weaknesses of {{tika-eval}} is the focus on alphabetic tokens...so,
by that metric we are now losing a small percentage of "common words" because
of the new exceptions. However, what we aren't able to detect easily...er...we
don't have a report for this...is the decrease in tokens that look like
"10,2,3,3,2,3,232342,23424" etc...
> Add a CSV/TSV detector
> ----------------------
>
> Key: TIKA-2833
> URL: https://issues.apache.org/jira/browse/TIKA-2833
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
> Attachments: csv_reports.zip
>
>
> Given initial experimentation, I think we can fairly easily add a fairly
> robust CSV/TSV detector that will identify well-formed (ha!) csvs and return
> the charset encoding and the delimiter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
