[jira] [Created] (TIKA-2279) Simplify token counting in tika-eval

[Tim Allison (JIRA)]

Tim Allison created TIKA-2279:
---------------------------------

             Summary: Simplify token counting in tika-eval
                 Key: TIKA-2279
                 URL: https://issues.apache.org/jira/browse/TIKA-2279
             Project: Tika
          Issue Type: Bug
          Components: tika-eval
            Reporter: Tim Allison
            Priority: Trivial


We can skip double analysis by moving the isAlphabeticOrIdeographic check into 
the CommonTokensCounter.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)