Rachelint commented on issue #12821: URL: https://github.com/apache/datafusion/issues/12821#issuecomment-2406458137
> > And I think maybe we can make clearer about when partial can help, and when partial will even get slower? > > In my mind the challenge with tweaking the "switch to partial mode" threshold setting is that some queries will likely get faster and some will likely get slower. If we can justify changing the default setting to some different constant I think it will be fine. However, if we are going to add more complex logic to decide when to switch modes in my opinion it needs to be significantly better than a static threshold (where significantly means "always better" or close to it) Got it, @jayzhan211 have tried some other values of `skip_partial_aggregation_probe_ratio_threshold` and `skip_partial_aggregation_probe_rows_threshold`, some queries seems improve obviously. And I have some thoughs like removing the `is_locked` field? Now, we take `skip_partial_aggregation_probe_rows_threshold` as a sample to define if we need to skip, when exceed we will not check this again). But I found some partial operator can get improvement from skipping, but have no chance to switch to due to `is_locked`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
