leaves12138 commented on code in PR #9807:
URL: https://github.com/apache/paimon/pull/9807#discussion_r4003024891
##########
paimon-python/pypaimon/read/scanner/bucket_select_converter.py:
##########
@@ -404,6 +404,17 @@ def __call__(self, *args) -> bool:
# forbids false-negatives.
return True
+ def may_contain(self, min_bucket: int, max_bucket: int, total_buckets:
int) -> bool:
+ """Conservatively test an inclusive manifest range without a known
partition."""
+ if min_bucket < 0 or max_bucket < min_bucket or total_buckets <= 0:
+ return True
+ try:
+ return any(min_bucket <= bucket <= max_bucket
+ for bucket in self._compute(None, total_buckets))
Review Comment:
Non-blocking Java parity note: this retains the existing Python selector's
conservative fallback rather than Java's full predicate-projection capability.
For example, with a BIGINT bucket key `id`, four buckets, and `(pt = 'alpha'
AND id = 1) OR (pt = 'omega' AND id = 7)`, Java's `BucketSelector` /
`PredicateProjectionConverter` can project the predicate to the bucket-key
condition and reject the manifest range `[0, 0]`; Python `may_contain(0, 0, 4)`
returns true because `_compute(None, 4)` falls back to all buckets. I
reproduced this against the Java classes built from this exact head and
confirmed the same fallback already exists in the base Python selector. This
only causes extra reads, not false-negative pruning, and matches the
conservative scope stated in the PR, so it does not block approval. Full
optimization parity would require a follow-up port of the inclusive bucket-key
predicate projection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
