sunchao commented on issue #5325: URL: https://github.com/apache/arrow-rs/issues/5325#issuecomment-1907422517
I think this code is for converting a `i32`/`i64` read from Parquet into an internal representation which uses big-endian for the byte order. Therefore, it is unrelated to the Parquet format spec. I don't remember exactly why we chose big-endian instead of little-endian for the above (it was added in https://github.com/sunchao/parquet-rs/pull/103). If one needs to writes out to Parquet after reading the decimals, then it might be better to use little-endian here to save some conversion cost. On the other hand, Java's `BigInteger` (used by Spark) uses big-endian so if for some reason if the decimal read here need to be consumed by Java, then it might be more efficient to use big-endian here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
