QuakeWang opened a new pull request, #8496:
URL: https://github.com/apache/paimon/pull/8496
### Purpose
Data evolution scans reused the full filter for row-range extraction, stats
pruning, and global index evaluation.
For mixed predicates such as `_ROW_ID = 1 OR f0 > 5`, the row-id visitor
cannot fully convert the predicate to row ranges. The old path could still
remove or push `_ROW_ID` branches into stats/global-index pruning, which may
narrow scan semantics, hit system-field handling bugs, or incorrectly enable
limit/top-n split pruning.
This PR keeps the full predicate for read-time filtering and limit/top-n
safety, while only pushing row-id-safe residual predicates into scan pruning
and global index evaluation. Python follows the same rule and excludes
`_ROW_ID` from stats predicates.
### Tests
- `mvn -pl paimon-core -am -DfailIfNoTests=false
-Dtest=DataEvolutionBatchScanTest,BtreeGlobalIndexTableTest,DataEvolutionTableTest#testMixedRowIdOrFilterDoesNotPruneByUnsafeStatsResidual+testMixedRowIdOrFilterDisablesUnsafeLimitPushDown+testLimitPushDownWithFilterShouldNotEarlyStop
test`
- `python -m pytest paimon-python/pypaimon/tests/range_test.py
paimon-python/pypaimon/tests/reader_predicate_test.py
paimon-python/pypaimon/tests/test_early_row_range_filter.py
paimon-python/pypaimon/tests/data_evolution_test.py paimon-python/pypaimon/
tests/predicates_test.py::PredicateTest::test_exclude_predicate_with_fields -q`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
