snizovtsev commented on issue #34539: URL: https://github.com/apache/arrow/issues/34539#issuecomment-1506265527
@westonpace thanks! I've tested your PR on my workload and stack overflows had gone. However Acero engine still tends to starve and leak memory in some cases. In latter case I can see that at some random point write speed degrades to almost 0 and the engine starts consuming memory until exhaustion. It has higher chances to finish successfully when number of output partitions is small like and reader is slow (after dropping system page cache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
