naman-modi commented on issue #22710: URL: https://github.com/apache/datafusion/issues/22710#issuecomment-5046250224
Hey @2010YOUY01, this refactor looks really interesting and I'd like to contribute. I went through the EPIC and the code, and can see #23657 adds spilling for the ordered partial/final streams and drops the `MemoryLimit::Finite` bail for those two `should_use_*` gates. Since the PR notes that ordered spilling is the groundwork the other modes reuse, the unordered paths (like should_use_partial_hash_stream, final_hash_stream, single_hash_stream) still fall back to `GroupedHashAggregateStream` under a memory limit. Is this something I can pick up: unordered Partial or Single spilling; reusing the ordered larger-than-memory logic from #23657 and dropping the `Finite` bail? If it collides with your in-flight work, pls point me at whatever's most helpful instead, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
