gitmodimo opened a new pull request, #46140: URL: https://github.com/apache/arrow/pull/46140
### Rationale for this change Current implementation of asof join does not respect backpressure from downstream nodes. PauseProducing implementation is empty. This leads to unbound memory consumption with slow sinks. ### What changes are included in this PR? With this change PauseProducing causes ProcessThread to pause when backpressure is applied. Pause logic stolen from source_node and seems to work. Inputs do not require additional handling sice each input has its own backpressure queue. ### Are these changes tested? Tested in my application. Still deciding how to test it. Either by checking memory consumption or by checking whether the pause is propagating upstream? The latter seems easier. ### Are there any user-facing changes? No PS Also I did not modify the single threaded version. Can someone using it chime in? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
