[
https://issues.apache.org/jira/browse/LUCENE-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doron Cohen updated LUCENE-1095:
--------------------------------
Attachment: lucene-1095-pos-incr.patch
{quote}
... it does not assume incoming tokens to have position-increment of 1.
So it should work ok when stop filters and "expanders" are concatenated.
{quote}
Well, chaining stop filters didn't work in previous patch.
Attached patch fixed this.
Problem was with tokenizers reuse of tokens (next(Token)).
CharTokenizer, KeywordTokenizer and StandardTokenizer
were fixed to reset positionIncrement to 1.
The way I see it *Tokenizers must* do this and *Filters must not* do this.
If there's no objection to this observation I'll javadoc it in Tokenizer
for future implementors.
> StopFilter should have option to incr positionIncrement after stop word
> -----------------------------------------------------------------------
>
> Key: LUCENE-1095
> URL: https://issues.apache.org/jira/browse/LUCENE-1095
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Hoss Man
> Assignee: Doron Cohen
> Attachments: lucene-1095-pos-incr.patch, lucene-1095-pos-incr.patch
>
>
> I've seen this come up on the mailing list a few times in the last month, so
> i'm filing a known bug/improvement arround it...
> StopFilter should have an option that if set, records how many stop words are
> "skipped" in a row, and then sets that value as the positionIncrement on the
> "next" token that StopFilter does return.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
