DarkWanderer commented on PR #9011: URL: https://github.com/apache/arrow-rs/pull/9011#issuecomment-3666538450
> Edit: I could see a world where users could opt-in to have ArrowFilter passed the dictionary in a pre-pass, and for this to then allow the reader to skip decoding dictionary encoded pages if there are no matches, but unless I'm remembering incorrectly this wouldn't allow skipping the IO... That is exactly what I am hoping for - to perform a multiple-range fetch of a few MB from `object_store` to filter down row groups to only ones I need, which saves me multiple gigabytes of actual S3 read. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
