ByteBaker commented on issue #6282: URL: https://github.com/apache/arrow-rs/issues/6282#issuecomment-2371742298
One more thing to add here. Parquet (and the entire arrow project) uses `flate2` with `rust_backend` feature enabled. Which uses `miniz` backend, thereby supporting level 10 compression, aka `UBER COMPRESSION`. Flate2 still chooses to call 9 as the `best` level of compression because with 10 we might run into performance issues on the user's device. The PR I created in `flate2` is merged. So the docs should mention this caveat very soon hopefully. But behaviorally speaking, using level = 10 in parquet shouldn't be a problem at all. Discretion is advised when using `flate2` separately. @alamb @tustvold what do you think? And should we close this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
