JerAguilon opened a new issue, #37796: URL: https://github.com/apache/arrow/issues/37796
### Describe the bug, including details regarding any error messages, version, and platform. I've encountered a subtle race condition in the asof join node that is particularly common for large parquet files with many row groups: 1. The left hand side of the asofjoin completes, so `InputFinished` proceeds as [expected](https://github.com/apache/arrow/blob/2455bc07e09cd5341d1fabdb293afbd07682f0b2/cpp/src/arrow/acero/asof_join_node.cc#L1323). So far so good 2. The right hand table(s) of the join are a huge dataset scan. They're still streaming and can legally still call `AsofJoinNode::InputReceived` ([ref](https://github.com/apache/arrow/blob/2455bc07e09cd5341d1fabdb293afbd07682f0b2/cpp/src/arrow/acero/asof_join_node.cc#L1689)) 3. If enough batches come in before execution stops, then we might exceed the `[high_threshold](https://github.com/apache/arrow/blob/2455bc07e09cd5341d1fabdb293afbd07682f0b2/cpp/src/arrow/acero/asof_join_node.cc#L575)` and tell the input node to pause via the [BackpressureController](https://github.com/apache/arrow/blob/2455bc07e09cd5341d1fabdb293afbd07682f0b2/cpp/src/arrow/acero/asof_join_node.cc#L540) 4. At this point, execution has stopped for the asofjoiner, so the right hand table(s) won't be dequeue'd, meaning `BackpressureController::Resume()` will never be called. I have hackily fixed this in a local checkout by storing an `atomic<bool>` of whether `EndFromProcessQueue` was called. If it turns `true`, then at [InputReceived](https://github.com/apache/arrow/blob/2455bc07e09cd5341d1fabdb293afbd07682f0b2/cpp/src/arrow/acero/asof_join_node.cc#L1682) I shortcircuit and return a `Status::OK()` without enqueueing the batch. Happy to submit a PR once I find bandwidth, but reporting this early in case others run into it. ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
