Alan Woodward created LUCENE-8395:
-------------------------------------
Summary: WordDelimiterGraphFilter can incorrectly add holes to a
TokenStream
Key: LUCENE-8395
URL: https://issues.apache.org/jira/browse/LUCENE-8395
Project: Lucene - Core
Issue Type: New Feature
Reporter: Alan Woodward
Assignee: Alan Woodward
Attachments: LUCENE-8395.patch
If a token consists entirely of delimiter characters, then
WordDelimiterGraphFilter will remove the token and insert a hole into the
TokenStream. However, it does this even if {{preserve_original}} is set, which
results in an extra gap being added to the stream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
