ump45nose commented on code in PR #2698:
URL:
https://github.com/apache/datafusion-ballista/pull/2698#discussion_r4586308961
##########
ballista/scheduler/src/state/mod.rs:
##########
@@ -540,6 +540,18 @@ impl<T: 'static + AsLogicalPlan, U: 'static +
AsExecutionPlan> SchedulerState<T,
error!(
"Failed to launch new task, could not get executor
metadata: {e}"
);
+ // Bound tasks were already marked Running at bind
+ // time. Return them to pending before refunding
+ // vcores, otherwise they hang forever (#2691).
+ let reset = state
+ .task_manager
+ .reset_unlaunched_tasks(&reservations)
+ .await;
+ if reset > 0 {
Review Comment:
Good point. I left this PR to the static/adaptive graphs to keep it focused,
and I would rather handle pipelined jobs in a follow-up: failing the job is a
bigger behaviour change than a reset, and it needs its own test with the
pipelined placement. Happy to open an issue for it if you agree. (AI-assisted
reply.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]