Jefffrey commented on PR #5154: URL: https://github.com/apache/arrow-rs/pull/5154#issuecomment-1835922634
For reference parquet-mr has similar logic, difference being instead of opting out of truncation for Decimal, they enforce that types need to opt-in: https://github.com/apache/parquet-mr/blob/97e6ba8e8343a7274e530a83496a7933cc5ffc01/parquet-column/src/main/java/org/apache/parquet/internal/column/columnindex/BinaryTruncator.java#L182-L225 Which has similar effect in ensuring Decimal isn't truncated (Float16 support isn't yet merged in for parquet-mr, but they aren't changing that logic in the Float16 PR so presumably will also not do truncation: https://github.com/apache/parquet-mr/pull/1142) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
