bujji8411 opened a new issue, #60328:
URL: https://github.com/apache/airflow/issues/60328
### Description
depends_on_past=True currently allows a task to depend only on the same task
in the previous dag_run.
There is no native way for a task to depend on the successful completion of
multiple specific tasks from the previous dag_run (within the same DAG).
Achieving this today requires custom sensors that directly query Airflow
metadata tables.
### Use case/motivation
In stateful or partially retryable pipelines, it is common to require that a
group of tasks in the previous dag_run has completed successfully before
continuing execution in the next run.
Example:
`A → B → C → D → E`
If:
dag_run_1 fails at task D
Then for dag_run_2: Task C should not start until C, D, and E from dag_run_1
have all succeeded
Today:
depends_on_past is insufficient (checks only the same task)
trigger_rule applies only within the same dag_run
Users must implement custom PythonSensors that query DagRun / TaskInstance
This adds boilerplate, DB coupling, and operational complexity for a fairly
common dependency pattern.
What would you like to happen?
Provide a declarative, first-class way for a task to depend on specific
tasks from the previous dag_run.
Illustrative example (API shape open for discussion):
```
C = PythonOperator(
task_id="C",
depends_on_past=True,
depends_on_previous_task_ids=["C", "D", "E"],
)
```
> Semantics:
In dag_run(N), task C is scheduled only if the listed tasks in dag_run(N-1)
all finished with success
First dag_run is not blocked
Backward compatible and fully opt-in
This would remove the need for custom sensors while keeping cross-run
dependencies explicit and readable.
### Related issues
No known existing issue covering this specific capability.
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
