Dandandan commented on PR #20481: URL: https://github.com/apache/datafusion/pull/20481#issuecomment-4288585249
> Hey @Dandandan — are you still planning to continue this RG-level morsel work? You mean morsel splitting, or? I plan to contribute to it. Current state: 1. Morsel planning landed in https://github.com/apache/datafusion/pull/21351 2. We want to do morsel splitting to further optimize queries with few row groups and partition skew at the "tail". We probably need some API changes in arrow-rs to make it work 3. @adriangb has some ideas to use it for better / adaptive row filters as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
