[
https://issues.apache.org/jira/browse/LUCENE-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527562#comment-16527562
]
Adrien Grand commented on LUCENE-8361:
--------------------------------------
We should disable some checks when the analysis chain includes one of these
token filters that truncate the token stream. If we made them correct so that
they consumed the wrapped token stream to eg. have access to the end offset, it
would defeat their purpose a bit, which is to reduce the amount of work to do.
> Make TestRandomChains check that filters preserve positions
> -----------------------------------------------------------
>
> Key: LUCENE-8361
> URL: https://issues.apache.org/jira/browse/LUCENE-8361
> Project: Lucene - Core
> Issue Type: Test
> Reporter: Adrien Grand
> Assignee: Alan Woodward
> Priority: Minor
> Attachments: LUCENE-8361.patch
>
>
> Follow-up of LUCENE-8360: it is a bit disappointing that we only found this
> issue because of a newly introduced token filter. I'm wondering that we might
> be able to make TestRandomChains detect more bugs by verifying that the sum
> of position increments is preserved through the whole analysis chain.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
