[
https://issues.apache.org/jira/browse/LUCENE-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501085
]
Toru Matsuzawa commented on LUCENE-902:
---------------------------------------
for stop word only "is".
sample words "A is B".
For instance,When Tokenizer on StopFilter returns the following as a result.
termText positionIncrement
"A" 1
"is" 1
"are" 0
"be" 0
"B" 1
The result of StopFilter.
termText positionIncrement
"A" 1
"are" 0
"be" 0
"B" 1
"A" and "are" and "be" become the same positions.
When thinking that it will process the result of a Japanese morphological
analysis with StopFilter, it becomes a problem.
> Check on PositionIncrement with StopFilter.
> --------------------------------------------
>
> Key: LUCENE-902
> URL: https://issues.apache.org/jira/browse/LUCENE-902
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 2.2
> Reporter: Toru Matsuzawa
> Attachments: stopfilter.patch
>
>
> PositionIncrement set with Tokenizer is not considered with StopFilter.
> When PositionIncrement of Token is 1, it is deleted by StopFilter. However,
> when PositionIncrement of Token following afterwards is 0, it is not deleted.
> I think that it is necessary to be deleted. Because it is thought same Token
> when PositionIncrement is 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
