[
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444908#comment-13444908
]
Chris Male commented on LUCENE-4343:
------------------------------------
+1 to the improvements and pursuing making it final.
> Clear up more Tokenizer.setReader/TokenStream.reset issues
> ----------------------------------------------------------
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
> Issue Type: Task
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to
> only describe a totally wacky case (CachingTokenFilter) and not the normal
> case.
> Ideally setReader would be final I think, but there are a few crazy
> tokenstreams to fix before I could make that work. Would also need something
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones
> that dont use MockTokenizer, which they really should), if we can do some
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> - private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> + // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't
> call reset()
> + private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding
> broken tests etc. We should see if we can be more thorough/ideally throw
> better exceptions when consumers are broken and its free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
