[
https://issues.apache.org/jira/browse/LUCENE-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741804#action_12741804
]
Yonik Seeley commented on LUCENE-1794:
--------------------------------------
bq. I am thinking of expanding this patch to include reset() impls for
state-keeping tokenizers/filters that do not currently have an analyzer...
+1
In the past we've always encouraged people to create their own analyzers by
plugging together the provided filters - we should keep this simple to do.
Now the problem: TokenStream.reset() has different semantics than what we are
using it for here. CachingTokenFilter uses it to start the replay of the last
string of tokens it saw (definitely bad for reuse).
So.... do we redefine reset()? Something like CachingTokenFilter is very
special case, and I don't feel like it should have it's own method. There are
other ways it could be implemented now anyway.
> implement reusableTokenStream for all contrib analyzers
> -------------------------------------------------------
>
> Key: LUCENE-1794
> URL: https://issues.apache.org/jira/browse/LUCENE-1794
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1794.patch, LUCENE-1794.patch
>
>
> most contrib analyzers do not have an impl for reusableTokenStream
> regardless of how expensive the back compat reflection is for indexing speed,
> I think we should do this to mitigate any performance costs. hey, overall it
> might even be an improvement!
> the back compat code for non-final analyzers is already in place so this is
> easy money in my opinion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
