Alan Woodward created LUCENE-8719:
-------------------------------------
Summary: FixedShingleFilter can miss paths at the end of a
TokenStream
Key: LUCENE-8719
URL: https://issues.apache.org/jira/browse/LUCENE-8719
Project: Lucene - Core
Issue Type: Bug
Reporter: Alan Woodward
Assignee: Alan Woodward
If we have a set of paths of differing lengths at the end of a TokenStream,
FixedShingleFilter will currently stop when the first path hits the end of the
stream, but this may miss extra longer paths. For example, the stream "a b:3/c
d e f" with a shingle size of 3 should produce paths "a b f", "a c d", "c d e"
and "d e f", but at the moment FixedShingleFilter will stop when it finds that
the path "b f" is too short to produce a shingle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
