bryanck opened a new pull request, #5198: URL: https://github.com/apache/iceberg/pull/5198
This PR fixes the vectorized reader for decimals that are fixed length binary and dictionary-encoded. Before these decimals would be downcast to 8 byte (long) precision. This only affects Parquet V2, as fixed length binary decimals aren't dictionary-encoded in Parquet V1. The Spark vectorized reader test for Parquet V2 was modified so this is now tested. Also included is a minor refactor of the big endian padding to share some common logic and make it easier to enhance later. In addition, the Spark benchmark test for the vectorized reader was enhanced to include both long encoded and fixed width binary encoded decimals. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
