2010YOUY01 commented on PR #15017: URL: https://github.com/apache/datafusion/pull/15017#issuecomment-2700720460
> Thank you @2010YOUY01 > > While this likely will result in slightly slower performance in some cases (as there is additional spilling) making sure the queries won't error seems like a very valuable property. Thank you for the review. For efficiency, it's possible to avoid the final spill by checking the buffered batch number and spilled file count, but this should better be done after we have accurate memory estimation https://github.com/apache/datafusion/issues/14748, I'll update the issue for this idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org