alamb commented on issue #1570: URL: https://github.com/apache/arrow-datafusion/issues/1570#issuecomment-1654747713
> After that is done, we plan to contribute back the work to DataFusion, but I just found this ticket. I am wondering what the current status of this ticket is. That sounds good > I am wondering if the community is willing to review yet another spilling implementation as it is one of the missing pices I am willing to review another spilling implementation for sure -- it is one of the missing features for datafusion to be a complete analytic solution To assit review, it would help to explain / document how it works (e.g. is it based on mmap, or are the intermediate groups spilled as arrow, or are they sorted?) ideally in code comments I made some diagrams here that might be helpful (but are mostly related to streaming groupby) https://docs.google.com/document/d/16rm5VR1nGkY6DedMCh1NUmThwf3RduAweaBH9b1h6AY/edit?usp=sharing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
