JingsongLi commented on PR #9884: URL: https://github.com/apache/paimon/pull/9884#issuecomment-5708354462
Good fix for a nasty failure mode. Verified the reasoning end to end: - The index path answers with all-or-nothing bitmaps and has no final-read filter, so an inexact scalar bitmap (empty when the scanner cannot evaluate anything, or a superset when only some conjuncts contributed) could not be corrected downstream. Routing those ranges to the raw search, where the exact final-read filter decides, is the right call. - Using `contributingFieldIds().containsAll(collectFieldIds(...))` as the exactness test is correct; `null` is used consistently as "cannot decide" and never as "no rows match". - The partial-coverage case re-scans the whole demoted range through raw. Since `ScoredGlobalIndexResult.or()` unions row ids, overlap with existing raw splits costs redundant work but cannot duplicate results. - `scalarPreFilter` is written and read on the read thread only, reset at the top of every `demoteUncoverableIndexSplits` call, and all four call sites invoke the demotion right after `splitSearchSplits` and before `preFilters`. The two new tests capture both failure shapes, deliberately with the closest row excluded — good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
