qlong opened a new issue, #15510: URL: https://github.com/apache/iceberg/issues/15510
### Feature Request / Improvement Iceberg supports writing shredded variant columns to Parquet (PR #14297). For a query like `WHERE variant_get(v, '$.price', 'int') > 100`, Iceberg can already skip Parquet files using manifest-level bounds, where the lower/upper bounds for $.price are stored as a Variant-encoded object on the v column's manifest entry (PR #15385). For better performance, Iceberg should also skip row groups within a file: when $.price is shredded, the physical Parquet column `v.typed_value.price.typed_value` has standard Parquet min/max statistics. Iceberg should use those stats to skip row groups where max < 100, the same way ParquetMetricsRowGroupFilter handles regular column predicates. The change is in parquet module, and should work with all engines. **Scope** - Add BoundExtract handling to ParquetMetricsRowGroupFilter - Map variant path → physical shredded Parquet column - Unshredded variant columns are not in scope (binary stats are not useful for path-based filtering) - Test with Spark ### Query engine None ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
