sweb opened a new pull request #9047: URL: https://github.com/apache/arrow/pull/9047
This PR adds capabilities to read decimal columns in parquet files that store them as i32 or i64. I tried to follow the approach in #8926 by using casts. However, there is an issue with my solution that I expect to be problematic: Casting from i32/i64 to decimal assumes that the integer value was intended to represent a decimal value, since it just transforms the i32/i64 to i128. This will lead to usability issues once someone tries to cast an actual `Int64Array` to `DecimalArray`, since this will most likely not return the expected outcome (I would expect that value to be multiplied by 10^scale). I am currently not sure how to resolve this while still using casts to get from i32/i64 to decimal. Maybe it makes sense to remove the casting operations and put the logic in `PrimitiveArrayReader::next_batch`. Let me know what you think! ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
