Simon Willnauer created LUCENE-4955:
---------------------------------------
Summary: NGramTokenFilter increments positions for each gram
Key: LUCENE-4955
URL: https://issues.apache.org/jira/browse/LUCENE-4955
Project: Lucene - Core
Issue Type: Bug
Components: modules/analysis
Affects Versions: 4.3
Reporter: Simon Willnauer
Fix For: 5.0, 4.4
NGramTokenFilter increments positions for each gram rather for the actual token
which can lead to rather funny problems especially with highlighting. if this
filter should be used for highlighting is a different story but today this
seems to be a common practice in many situations to highlight sub-term matches.
I have a test for highlighting that uses ngram failing with a StringIOOB since
tokens are sorted by position which causes offsets to be mixed up due to ngram
token filter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
