sankarreddy-atlan opened a new pull request, #3537: URL: https://github.com/apache/iceberg-rust/pull/3537
## Which issue does this PR close? - Closes #3536. - Refs #3464. ## What changes are included in this PR? `MinMaxColAggregator` now uses truncated string and binary statistics. A truncated min is a prefix, and parquet-rs increments a truncated max, so both are valid bounds. These cases leave the column with no file bound on that side: - A truncated value of another type, such as a `fixed` longer than 64 bytes. Its length is wrong. - A row group without statistics, as in Java's [`ParquetMetrics`](https://github.com/apache/iceberg/blob/2a6c556c0c883c04c6d3fbd68e6aeda36b91e0aa/parquet/src/main/java/org/apache/iceberg/parquet/ParquetMetrics.java#L315-L319). Long strings in single row group files now get the 64-byte truncated bounds. This does not fix the position delete `file_path` case in #3464, because the lower and upper bounds are still not equal. ## Are these changes tested? Yes. - `test_truncated_statistics_bound_the_file` writes two row groups. A 100-byte string is the min of one and another is the max of the other. It fails on main with lower bound `"b"`. - Two aggregator unit tests: a truncated `fixed` column gets no bounds, even when a later row group has an exact max. A row group without statistics drops both bounds, even when a later row group has values. `ParquetWriter` cannot write a file where only some row groups have statistics, so the writer loop's call to `drop_bounds` has no end-to-end test. ## AI Disclosure This PR was prepared with AI assistance and reviewed/tested by the author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
