[
https://issues.apache.org/jira/browse/LUCENE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164671#comment-13164671
]
Dawid Weiss commented on LUCENE-3298:
-------------------------------------
Sudarshan,
If you take a look at the trunk version of FSTLookup it uses FSTCompletion
underneath and that class in turn stores arbitrary byte sequences (text is
converted to UTF8). Not byte outputs, but you could create your "suggestions"
by concatenating input with output, divided with a marker or something. This
will bloat the automaton, but if your data is relatively small, it's not a
problem and you can still extract your "outputs" after suggestions are
retrieved from the FST. Take a look at FSTCompletion and FSTCompletionBuilder
(and tests), they'll be helpful.
> FST has hard limit max size of 2.1 GB
> -------------------------------------
>
> Key: LUCENE-3298
> URL: https://issues.apache.org/jira/browse/LUCENE-3298
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/FSTs
> Reporter: Michael McCandless
> Priority: Minor
> Attachments: LUCENE-3298.patch
>
>
> The FST uses a single contiguous byte[] under the hood, which in java is
> indexed by int so we cannot grow this over Integer.MAX_VALUE. It also
> internally encodes references to this array as vInt.
> We could switch this to a paged byte[] and make the far larger.
> But I think this is low priority... I'm not going to work on it any time soon.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
