Bruno Roustant created LUCENE-9064:
--------------------------------------
Summary: Can we remove the FST cache in Kuromoji and Nori
analyzers?
Key: LUCENE-9064
URL: https://issues.apache.org/jira/browse/LUCENE-9064
Project: Lucene - Core
Issue Type: Improvement
Reporter: Bruno Roustant
Is the ~30k han cache in kuromoji redundant after LUCENE-8920?
[https://github.com/apache/lucene-solr/blob/813ca77250db29116812bc949e2a466a70f969a3/lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/TokenInfoFST.java#L35-L38])
The entire linked file's purpose is all around this caching, so if its not
needed anymore it would be a nice cleanup. But it was definitely needed for
good performance before, so we shoudl be careful. Nori analyzer has the exact
same thing (file has the same name) for ~10k hangul syllables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
