zanmato1984 commented on PR #45268: URL: https://github.com/apache/arrow/pull/45268#issuecomment-2595171788
I'm thinking this might not be very necessary for the following two reasons: 1. I think in the current acero design, once an error occurs in a task, this error will be propagated to the upmost `ExecPlan`, which will in turn call the `Abort` method of task scheduler, which will finish all the running tasks regardless of an individual task group's finished count being arguably "wrong". 2. Even if we think the task group's finished count as "wrong" (once errors occur in a task group), and we want to make it "right" so that the task group will finish in a graceful way (it doesn't depend on the `Abort`), we have a problem of invoking the continuation. Generally a continuation of a task group shouldn't be called if some tasks of the group meet error - this is the case for current implementation because the task group is not considered "finished" (thanks to the "wrong" finished count). But this will change if we make the finished count "right" - the continuation will be (undesirably) invoked. What do you think? @wuzhoupei -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
