tustvold commented on issue #1652: URL: https://github.com/apache/arrow-datafusion/issues/1652#issuecomment-1023655722
> skip the whole row group if no keys in the dictionary match. You'd need to check that column chunk hadn't spilled, but yes that is broadly speaking the access pattern I wish to support with the work in https://github.com/apache/arrow-rs/issues/1191. Specially exploit https://github.com/apache/arrow-rs/pull/1180 to cheaply get dictionary data, evaluate predicates on it, and use this to construct a refined row selection to fetch. As @alamb alludes to the timelines for this effort are not clear, and there may be other more pressing things for me to spend time on, but it is definitely something that I _hope_ to deliver in the next month or so. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
