GitHub user westonpace added a comment to the discussion: Future plans for Acero
The sidecar thread in the asof join node was an implementation choice by the original author. It's possible to get rid of it but you would need to do a bit of work to refactor the asof join node implementation so that work is done by the push. It's been a long while since I looked at it but I think the logic is basically that each side pushes its data into a queue and then the sidecar thread is constantly waking up to see if there is enough data to form rows and, if so, creates and publishes those rows. You could have a solution where the work done by the sidecar thread is done after the batch is pushed into the queue. GitHub link: https://github.com/apache/arrow/discussions/47331#discussioncomment-14221768 ---- This is an automatically sent email for user@arrow.apache.org. To unsubscribe, please send an email to: user-unsubscr...@arrow.apache.org
