milenkovicm commented on issue #9359: URL: https://github.com/apache/arrow-datafusion/issues/9359#issuecomment-2051695693
FWIW, I believe I was looking at this problem some time ago, if I remember correctly the issue was that one of the memory consumer, presumably `ExternalSorterMerge` was returning all the memory it allocated just to ask for new allocation with similar size very soon after. `MemoryConsumer` would return all the memory when dropped. My suspect is: https://github.com/apache/arrow-datafusion/blob/a5cf0b8902ae55b81ac86b875c7e94cf1bdc205d/datafusion/physical-plan/src/sorts/sort.rs#L263 or https://github.com/apache/arrow-datafusion/blob/a5cf0b8902ae55b81ac86b875c7e94cf1bdc205d/datafusion/physical-plan/src/sorts/sort.rs#L359 or somewhere around it. I cant find more information looks like that branch is MIA -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
