2010YOUY01 commented on issue #12919: URL: https://github.com/apache/datafusion/issues/12919#issuecomment-2413572118
> The underlying problem we're having is lots of queries seem to exceed the memory pool limits, `generate_series` is just one example of the problem. I have noticed a similar issue with external aggregation https://github.com/apache/datafusion/issues/12937 Now only Aggregation/Sort/Sort-merge-join supports spilling, it's likely they didn't actually free memory consumption after writing intermediates to disk Other accumulating operators should keep track of memory, and return error if it exceeds memory budget Do you have any idea what kind of query/operators might not respect memory limit? 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
