Rachelint commented on issue #24704: URL: https://github.com/apache/datafusion/issues/24704#issuecomment-6079150930
> I do think it could lower the peak memory requirements for queries with high cardinality aggregates... Seems in final when bucketing is on, the memory peak will just be `1.015 * state size` rather than original `2 * state size`; and when bucketing is off, the memory usage is always low? Maybe it is already good enough for reducing memory usage? > I think the Utf8 case is still open after https://github.com/apache/datafusion/pull/26116 (Partial) and https://github.com/apache/datafusion/pull/25724 (Final) Yes, for utf8 or more complex type, bucketing seems still lead to slight performance regression, I think we should continue to optimize. But if take its effect of memory usage reducing into consider, the slight regression seems acceptable (compared with blocked approach) ? And it is actually much simpler than blocked approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
