alamb commented on issue #7489: URL: https://github.com/apache/arrow-rs/issues/7489#issuecomment-2936283780
> @etseidl @alamb it seems like the issue is fixed now! :) Now that https://github.com/apache/arrow-rs/pull/7555 is merged, I checked out arrow and parquet with the recent git version: @etseidl does great work :) > The file sizes seem like there really should be a statistics_truncate_length default somewhere below or equal to 1024 or so. Yes I agree -- here is a PR that proposes defaulting it to 64 (to match page stats). Any thoughts? - https://github.com/apache/arrow-rs/issues/7490 - https://github.com/apache/arrow-rs/pull/7578 > @alamb a slightly offtopic question: Comparing EnabledStatistics CHUNK and PAGE (with statistics_truncate_length == None) it seems that in PAGE mode compared to CHUNK mode, the file is 3 times the size? Yes, it is currently 3 times the size. As @etseidl mentions, we have a plan to fix this: - https://github.com/apache/arrow-rs/issues/7580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
