adriangb commented on PR #9972: URL: https://github.com/apache/arrow-rs/pull/9972#issuecomment-4467424069
@etseidl after much profiling, debugging, etc. I've been able to get this to work with no performance impact (within noise). I recognize this is a non-trivial change but it introduces no public APIs and in theory if it is problematic in any way we can back out of it. The benefit of doing things this way is that we automatically patch buggy / problematic page size blowouts for everyone, without code changes needed on their end or guessing of column sizes necessary. One thing we could do to derisk if you want: add a config option to disable this behavior. Thanks for reviewing this, I hope we can make it work 😄 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
