[
https://issues.apache.org/jira/browse/SOLR-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13058489#comment-13058489
]
Dawid Weiss commented on SOLR-2628:
-----------------------------------
Yes, this may be the case. It'd need to be investigated because storing words
in a hashtable will also bump memory requirements, whereas an FST can at least
reuse some prefixes and suffixes.
> use of FST for SynonymsFilterFactory and synonyms.txt
> -----------------------------------------------------
>
> Key: SOLR-2628
> URL: https://issues.apache.org/jira/browse/SOLR-2628
> Project: Solr
> Issue Type: New Feature
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Environment: Linux
> Reporter: Bernd Fehling
> Assignee: Dawid Weiss
> Priority: Minor
> Labels: suggestion
>
> Currently the SynonymsFilterFactory builds up a memory based SynonymsMap.
> This can generate huge maps because of the permutations for synonyms.
> Now where FST (finite state transducer) is introduced to lucene this could
> also be used for synonyms.
> A tool can compile the synoynms.txt file to a binary automaton file which can
> then be used
> with SynoynmsFilterFactory.
> Advantage:
> - faster start of solr, no need to generate SynonymsMap
> - faster lookup
> - memory saving
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
