mohitgurav20 commented on PR #26000: URL: https://github.com/apache/datafusion/pull/26000#issuecomment-6271311518
Thanks @mkleen — this decision tree is much clearer. I'll rework the rule to follow it: 1. No GROUP BY → rewrite only when multi-partition and not a cheap count(DISTINCT tiny/smallint) 2. With GROUP BY → skip when the distinct agg has a fast GroupsAccumulator (e.g. count(DISTINCT integer)); otherwise rewrite by default 3. Config option only on the last case, so users can skip the rewrite when memory matters more than speed I'll update the PR with this logic + tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
