JerAguilon commented on code in PR #37839:
URL: https://github.com/apache/arrow/pull/37839#discussion_r1336515512
##########
cpp/src/arrow/acero/asof_join_node.cc:
##########
@@ -579,7 +581,16 @@ class BackpressureHandler {
}
}
+ Status ForceShutdown() {
+ // It may be unintuitive to call Resume() here, but this is to avoid a
deadlock.
+ // Since acero's executor won't terminate if any one node is paused, we
need to
+ // force resume the node before stopping production.
+ backpressure_control_->Resume();
+ return input_->StopProducing();
Review Comment:
Yep.
As an aside, I feel like a more invasive change could fix this issue in the
general case. If a node (in this example asof join) has:
* Called `output->InputFinished()` AND
* Called `output_->InputReceived` for however many record batches it
advertised on `InputFinished`
We should be able to shut down execution, even if the node's inputs:
* are paused or
* not done streaming
* haven't called `InputFinished`
But I think this is a more invasive change to `exec_plan.h` and might have
some hairy issues that I'm not thinking of.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
