plusplusjiajia opened a new pull request, #642:
URL: https://github.com/apache/paimon-rust/pull/642
### Purpose
_ROW_ID is not a table column, so callers hand-build the leaf with a
placeholder index (0, as test_with_filter_extracts_row_id_ranges does).
ReadBuilder::with_filter ran normalize_filter — whose
partition/bucket split matches by index — before that leaf was removed, so
on a table whose first field is a partition key, WHERE _ROW_ID >= 10 became a
predicate on that key and pruned partitions by comparing
them against row ids, silently dropping rows.
Java strips row-id conjuncts by name in
DataEvolutionBatchScan.rowIdSafeResidualFilter; pypaimon rejects a field not in
partition_keys before looking at an index. Rust was the only implementation
matching by
index with no name-level guard.
### Brief change log
- strip_row_id_filter removes row-id conjuncts by name and feeds only
normalize_filter.
- Its complement retain_row_id_filter puts them back as a data predicate
when extract_row_id_ranges has no range form — leaf_to_ranges covers
Eq/Gt/GtEq/Lt/LtEq/In only, so BETWEEN, !=, NOT IN and mixed
disjunctions must still filter rows.
- Replaces remove_row_id_filter, which kept a whole Or mentioning _ROW_ID
and left the leaf in place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
