anoopj opened a new issue, #3482: URL: https://github.com/apache/iceberg-rust/issues/3482
### Apache Iceberg Rust version main @697abce ### Describe the bug Partition data and column bounds in a manifest are decoded using that manifest's own embedded (pre-promotion) schema, so they carry the narrower type (e.g. `int`). The scan predicate, however, is bound to the table's current schema, so its literal carries the promoted type (e.g. `long`). Datum's `PartialOrd/PartialEq` only compare when both the literal variant and the primitive type match (. Comparing an Int value against a Long literal therefore yields None, which the pruning visitors read as "cannot match" and drop the file. This affects partition tuple pruning as well as bounds-based pruning. The other two stages are already correct: - `ManifestEvaluator` decodes bounds on demand with the predicate's type via the length-aware Datum::try_from_bytes, which widens a 4-byte int bound to long. - `RowGroupMetricsEvaluator` fails open, so it loses some pruning but never drops rows. This is the same class of silent-prune bug as #3314 (INT32/INT64-encoded decimal page bounds). ### To Reproduce 1. Create a table partitioned by identity on an int column. 2. Write a data file whose value for that column is, say, 5. 3. Promote the column int -> long (schema evolution). 4. Scan with a filter on that column, e.g. col >= 5 (also reproducible with col > 4, col IN (5, 6)). Actual: zero rows because the data file is pruned during planning. Trivial illustration of the underlying mismatch: ```rust // returns None, expected Some(Ordering::Equal) Datum::int(5).partial_cmp(&Datum::long(5)); ``` ### Expected behavior Expected: the row is returned. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
