[
https://issues.apache.org/jira/browse/LUCENE-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2211:
----------------------------------
Attachment: LUCENE-2211.patch
More updates to TeeSink and also BaseTokenStreamTestCase to still call
clearAttributes to force TS to not reuse attributes from previous calls to
incrementToken(). This was one of the first traps, Robert investigated in 2.9.
> Advances BaseTokenStreamTestCase that uses a fake attribute to check, if
> clearAttributes() was called correctly - found bugs in contrib/analyzers
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2211
> URL: https://issues.apache.org/jira/browse/LUCENE-2211
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis, contrib/analyzers
> Affects Versions: 2.9, 2.9.1, 3.0
> Reporter: Uwe Schindler
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2211.patch, LUCENE-2211.patch, LUCENE-2211.patch,
> LUCENE-2211.patch, LUCENE-2211.patch, LUCENE-2211.patch
>
>
> Robert had the idea to use a fake attribute inside BaseTokenStreamTestCase
> that records if its clear() method was called. If this is not the case after
> incrementToken(), asserTokenStreamContents fails. It also uses the attribute
> in TeeSinkTokenFilter, because there a lot of copying, captureState and
> restoreState() is used. By the attribute, you can track wonderful, if
> save/restore and clearAttributes is correctly implemented. It also verifies
> that *before* a captureState() it was also cleared (as the state will also
> contain the clear call). Because if you consume tokens in a filter, capture
> the consumed tokens and insert them, the capturedStates must also be cleared
> before.
> In contrib analyzers are some test that fail to pass this additional
> assertion. They are not fixed in the attached patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
