2010YOUY01 commented on PR #15017:
URL: https://github.com/apache/datafusion/pull/15017#issuecomment-2700720460

   > Thank you @2010YOUY01
   > 
   > While this likely will result in slightly slower performance in some cases 
(as there is additional spilling) making sure the queries won't error seems 
like a very valuable property.
   
   Thank you for the review. 
   For efficiency, it's possible to avoid the final spill by checking the 
buffered batch number and spilled file count, but this should better be done 
after we have accurate memory estimation 
https://github.com/apache/datafusion/issues/14748, I'll update the issue for 
this idea.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org

Reply via email to