[
https://issues.apache.org/jira/browse/LUCENE-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746429#action_12746429
]
Michael Busch commented on LUCENE-1842:
---------------------------------------
Tim, why can't you achieve the same by implementing an Analyzer with
reusableTokenStream(). You could have a class holding all streams/tokenizers
that you put into your own ThreadLocal. Then of course you need an instance of
each tokenizer/stream/filter per thread - is that what you're trying to avoid?
You create them all once per thread, so their AttributeSource never has to
change. You can still change for every document (field) which input streams
your merging tokenizer uses, as long as it takes them from your ThreadLocal.
One other question: does you merging tokenizer know before it can inspect the
tokens returned from its inputs which one it will itself return? The answer
must be yes, otherwise you can't avoid copying in any case, right?
If this comment doesn't make any sense to you then I still didn't understand
what you're trying to achieve. :)
> Add reset(AttributeSource) method to AttributeSource
> ----------------------------------------------------
>
> Key: LUCENE-1842
> URL: https://issues.apache.org/jira/browse/LUCENE-1842
> Project: Lucene - Java
> Issue Type: Wish
> Components: Analysis
> Reporter: Tim Smith
> Priority: Minor
>
> Originally proposed in LUCENE-1826
> Proposing the addition of the following method to AttributeSource
> {code}
> public void reset(AttributeSource input) {
> if (input == null) {
> throw new IllegalArgumentException("input AttributeSource must not be
> null");
> }
> this.attributes = input.attributes;
> this.attributeImpls = input.attributeImpls;
> this.factory = input.factory;
> }
> {code}
> Impacts:
> * requires all TokenStreams/TokenFIlters/etc to call addAttribute() in their
> reset() method, not in their constructor
> * requires making AttributeSource.attributes and
> AttributeSource.attributesImpl non-final
> Advantages:
> Allows creating only a single actual AttributeSource per thread that can then
> be used for indexing with a multitude of TokenStream/Tokenizer combinations
> (allowing utmost reuse of TokenStream/Tokenizer instances)
> this results in only a single "attributes"/"attributesImpl" map being
> required per thread
> addAttribute() calls will almost always return right away (will only be
> "initialized" once per thread)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
