goingforstudying-ctrl opened a new pull request, #2192:
URL: https://github.com/apache/datafusion-ballista/pull/2192
## Which issue does this PR close?
Closes #2150.
## Rationale for this change
When the adaptive query planner replans after a stage completes, it may
decide that a stage which is **still running** is no longer needed — e.g. the
build side of a join produced no data, so the probe side is moot. The planner
correctly returns those stage ids as "cancelled", but the scheduler never acted
on that list: it only logged
```
there are stages to be cancelled but its not implemented. stages to cancel:
[...]
```
The `Running` stage stayed in the graph, so the job kept waiting on it and
could sit forever, and a late task completion from the cancelled stage failed
the whole status update with `Invalid stage ID ... is not running`.
## What changes are included in this PR?
- **`AdaptiveExecutionGraph::update_stage_progress`** now retires each stage
the replan cancels, via a new `retire_cancelled_stage`:
- a `Running` stage is moved to `Failed(TaskKilled)` (cancelling in-flight
tasks) so the job no longer waits on it;
- a `Resolved`/`UnResolved` stage is dropped outright (it has no in-flight
tasks);
- a terminal (`Successful`/`Failed`) stage is left untouched.
- **`update_task_status`** treats a late task status for a retired
(`Failed`) stage as stale and discards it with a warning instead of erroring,
so the rest of the batch still processes.
- **`TaskManager`** no longer aborts the entire grouped update when one
stage id is unexpected — it logs and continues with the remaining stages.
- Added a lifecycle test
(`test_replan_cancelled_stage_is_retired_and_late_task_discarded`) reproducing
the orphan-stage scenario end to end.
## Are these changes tested?
Yes. New unit test drives a join through `revive`, completes the build side
with empty output (triggering the replan that cancels the probe stage), then
asserts the probe stage is retired and that a late task completion from it is
discarded rather than failing the update.
Full `state::` suite: **195 passed, 0 failed**. `cargo clippy -p
ballista-scheduler --lib` and `cargo fmt` are clean.
## Are there any user-facing changes?
No API changes; the scheduler no longer wedges on AQE-cancelled stages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
