zhuqi-lucas commented on PR #20481: URL: https://github.com/apache/datafusion/pull/20481#issuecomment-4288944667
Thanks @Dandandan! That makes sense. Once morsel splitting lands, I'd like to build dynamic RG pruning for TopK on top of it — when a morsel is pulled from the queue, check the RG's min/max statistics against the current TopK dynamic filter threshold and skip it entirely if it can't contain qualifying rows. This is tracked in #21399. Happy to collaborate when the morsel splitting API is ready! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
