steveloughran commented on PR #15629: URL: https://github.com/apache/iceberg/pull/15629#issuecomment-4566950412
Marking as ready for review to the extent that the current test dataset shows speedup when ran atop the separate PRs needed for predicate pushdown on shredded variant fields. here each sql query is run with avro, unshredded and shredded variants: <img width="346" height="51" alt="Screenshot 2026-05-28 at 19 08 52" src="https://github.com/user-attachments/assets/1a977563-5253-40db-b6e3-6f9c8c43411a"; /> before: note how shredded is pathologically worse <img width="1307" height="1114" alt="Screenshot 2026-05-28 at 19 06 48" src="https://github.com/user-attachments/assets/361c755d-c12e-46db-b247-5322a6ad446a"; /> after: note how shredded variants are faster than unshredded for filtering operations. <img width="1275" height="972" alt="Screenshot 2026-05-28 at 19 08 15" src="https://github.com/user-attachments/assets/a1001c72-7882-4e7e-bb54-76b1328bfc09"; /> Special highlight for filtering on an integer field where the value is out of range or not in the IN membership of a query. These only examine the metrics and can skip reading any of the values. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
