vismaytiwari opened a new pull request, #16384: URL: https://github.com/apache/lucene/pull/16384
### Description Fixes #14137. With a valid analyzer chain — `StandardTokenizer` → `WordDelimiterGraphFilter(GENERATE_WORD_PARTS | GENERATE_NUMBER_PARTS | CATENATE_NUMBERS)` → `FlattenGraphFilter` → `FixedShingleFilter` — indexing input such as `555,0` throws `IllegalArgumentException: first position increment must be > 0 (got 0)`. This is the same chain Elasticsearch's `search_as_you_type` uses. `FixedShingleFilter` took a shingle's position increment from the emitting base token. But when earlier base positions produce no shingle (their graph routes are too short to fill the window) their increments were dropped, and a base reached as a graph alternative can itself carry a `posInc` of 0 — so the stream's first shingle could be emitted with `posInc=0`, which the indexer rejects. For `555,0` the flattened graph feeding the filter is `"5550"`(posInc=1), `"555"`(posInc=0), `"0"`(posInc=1). The `"5550"` base can't form a bigram and emits nothing, so the first emitted shingle `"555 0"` is based at `"555"` (posInc=0) and inherited that 0. The fix accumulates the base-position increments across bases that emit nothing and applies the total to the next emitted shingle (0 for further graph routes stacked at the same base). So `"555 0"` now carries `1 + 0 = 1`. Normal streams are unchanged: a single base still contributes its own increment, and stacked graph routes at the same base still get 0. Added `TestFixedShingleFilter#testGraphInputDoesNotEmitLeadingZeroPositionIncrement`, which builds the reported chain over `555,0`. It fails on `main` (leading `posInc=0`) and passes with this change. Full `TestFixedShingleFilter` and `./gradlew tidy` are green. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
