[
https://issues.apache.org/jira/browse/LUCENE-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741859#action_12741859
]
Yonik Seeley commented on LUCENE-1794:
--------------------------------------
It depends on the use case for CachingTokenFilter.
When it's used in places like QueryParser.getFieldQuery(), the consumer creates
the CachingTokenFilter and can rewind it too.
If one has managed to use the same instance more than once in the same
document, other tricks could be used such as resetting to the beginning after
false is returned from incrementToken() or implementing rewind in end(). Seems
like either would work.
But in reality, the concept of CachingTokenFilter isn't really compatible with
the concept of reuse at all... so I don't think we necessarily need to do
anything except document that it's not reusable. Adding rewind() to
TokenStream won't solve this semantic problem.
> implement reusableTokenStream for all contrib analyzers
> -------------------------------------------------------
>
> Key: LUCENE-1794
> URL: https://issues.apache.org/jira/browse/LUCENE-1794
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1794.patch, LUCENE-1794.patch, LUCENE-1794.patch
>
>
> most contrib analyzers do not have an impl for reusableTokenStream
> regardless of how expensive the back compat reflection is for indexing speed,
> I think we should do this to mitigate any performance costs. hey, overall it
> might even be an improvement!
> the back compat code for non-final analyzers is already in place so this is
> easy money in my opinion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
