DictionaryCompoundWordTokenFilter fails to create some tokens for final parts
of words
--------------------------------------------------------------------------------------
Key: LUCENE-3038
URL: https://issues.apache.org/jira/browse/LUCENE-3038
Project: Lucene - Java
Issue Type: Bug
Components: Analysis
Affects Versions: 3.1, 4.0
Reporter: Filip Svendsen
Fix For: 4.0, 3.1
DictionaryCompoundWordTokenFilter: Due to an off-by-one error, a word component
placed last in a compound word, will not get a token if its length is equal to
the minimal sub-word length.
Example:
min sub-word length: 4
Dictionary: {"alfa", "beta"}
word: "alfabeta"
Created tokens: {"alfabeta", "alfa"}
Expected tokens: {"alfabeta", "alfa", "beta"}
I have a patch with a testcase that fails on versions 3.1 and 4.0 (probably for
everything between as well, and for previous versions), along with a bugfix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
