LuciferYang commented on issue #9883: URL: https://github.com/apache/paimon/issues/9883#issuecomment-5710616065
Scope update: #9909 fixes the fully-unevaluable half of this issue (the scalar index cannot evaluate the predicate at all, so the old empty bitmap dropped every index-covered row). It intentionally does not close this issue. Still open here: a partially-evaluable predicate keeps the index fast path and can return a wrong top-K, because the scalar evaluator drops the unsupported conjunct and returns a non-null superset. Two shapes: - different fields, e.g. `a = 1 AND tags IS NOT NULL` where the array conjunct is unsupported; - same field, e.g. `id >= 0 AND CAST(id AS BIGINT) > 0` where the CAST conjunct is unsupported. Field coverage (`contributingFieldIds`) is not sufficient to prove exactness: the second shape covers every referenced field yet is still inexact. A fix needs an expression-level exactness signal or residual predicate from the evaluator, then either raw fallback or candidate re-checking with extra retrieval to refill the filtered top-K (filtering only the initial K would still miss qualifying rows displaced from that candidate set). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
