Jefffrey commented on code in PR #5003:
URL: https://github.com/apache/arrow-rs/pull/5003#discussion_r1390297511
##########
parquet/src/column/writer/mod.rs:
##########
@@ -1170,6 +1188,7 @@ fn increment_utf8(mut data: Vec<u8>) -> Option<Vec<u8>> {
mod tests {
use crate::{file::properties::DEFAULT_COLUMN_INDEX_TRUNCATE_LENGTH,
format::BoundaryOrder};
Review Comment:
Actually in regards to truncation, is it intended to disallow truncation of
f16 stats or it should take place anyway?
Nowhere in the new spec for f16 does it mention this case, and neither did I
see relevant changes in https://github.com/apache/arrow/pull/36073, but perhaps
I missed it?
e.g. if user set the truncation for column index length to 1 byte, should we
still truncate f16 to one byte since its underlying representation if Fixed len
byte array, or should leave it as 2 bytes as truncating f16 doesn't make sense
since it doesn't follow the sort order for fixed len byte arrays?
@benibus
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
