marvinlanhenke commented on issue #10752: URL: https://github.com/apache/datafusion/issues/10752#issuecomment-2150024521
I did some digging in order to find out why / or where the writing of those statistics is not supported (yet). Since I'm not familiar with the parquet impl, here are my findings, which might be useful in a follow-up ticket in arrow-rs. 1. When trying to `fn write_slice()` the min, max values are never updated due to a [filter-condition;](https://github.com/apache/arrow-rs/blob/master/parquet/src/column/writer/encoder.rs#L137-L146) that checks if the type is INTERVAL 2. In order to support updating the min max values, we need to handle the comparison of INTERVAL [here](https://github.com/apache/arrow-rs/blob/master/parquet/src/column/writer/mod.rs#L1122-L1167) I think this should be possible, or put differently, I don't see the reason yet, why this is not supported? Somethin similar (comparing FixedLenByteArrays) is already done for DECIMAL [here](https://github.com/apache/arrow-rs/blob/master/parquet/src/column/writer/mod.rs#L1200)? Perhaps, you have some more information on this @alamb - otherwise this might be enough information to file a ticket in arrow-rs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
