Hi, Futures run callbacks on the thread that marks then as finished. It seems that inside of the Source node’s generator loop does add a callback (https://github.com/iChauster/arrow/blob/asof_join2/cpp/src/arrow/compute/exec/source_node.cc#L130 <https://github.com/iChauster/arrow/blob/asof_join2/cpp/src/arrow/compute/exec/source_node.cc#L130>) which continues the loop. I’m not entirely sure myself how this code works (this generator + control flow thing is very opaque), but my guess is that’s what’s causing it.
One further note, copying a Future<> actually maintains a reference to the same underlying future, which may also be unexpected at first. Specifically in your code, doing Future<> to_finish = backpressure_future_; to_finish.MarkFinished(); is equivalent to just backpressure_future_.MarkFinished(). Anyway, I wouldn’t currently rely on PauseProducing/ResumeProducing unfortunately. It is currently not tested anywhere as far as I can tell and ignored by a lot of nodes (such as HashJoinNode). Michal and I have some work in progress involving a new scheduler with first-class support for back pressure. Sasha > On Jul 20, 2022, at 1:49 PM, Ivan Chau <ivan.m.c...@gmail.com> wrote: > > backpressure_future_
