premsharma-IH opened a new issue, #67462: URL: https://github.com/apache/airflow/issues/67462
### Description ### Description Currently, Apache Airflow provides convenient UI options to clear and re-run failed tasks for a DAG run. However, there is no equivalent option to selectively re-run only tasks in the `skipped` state. This becomes operationally challenging for large dynamic DAGs and TaskGroups where hundreds of tasks may exist. ### Use Case We have a dynamically generated DAG where: * tasks are created from metadata/configuration stored externally * each task represents ingestion/loading for a table * DAG contains 500+ dynamically generated tasks inside TaskGroups * tasks execute in parallel using pools and concurrency controls In some operational scenarios, tasks may enter the `skipped` state due to: * transient resource limitations * scheduler timing conditions * dag run timeout * temporary orchestration issues Currently Airflow allows: * re-running failed tasks * clearing entire DAG runs * manually selecting tasks one-by-one However, there is no simple mechanism to: > re-run only skipped tasks ### Problem For large DAGs, manually selecting skipped tasks is operationally difficult and error-prone. Re-running the full DAG is also inefficient because: * successful tasks do not need re-execution * only skipped tasks require recovery * compute and runtime costs increase unnecessarily ### Proposed Enhancement Add a UI and/or CLI capability to: * filter tasks by `skipped` state * clear and re-run only skipped tasks Possible examples: * "Clear Skipped Tasks" * "Re-run Skipped Tasks" * additional checkbox alongside existing failed/upstream_failed options ### Benefits * Improves operational recovery workflows * Reduces unnecessary task re-execution * Helpful for dynamic DAGs and TaskGroups * Improves usability for large-scale orchestration pipelines ### Environment * Apache Airflow 2.7.2 * AWS MWAA * Dynamic DAG generation using TaskGroups * Parallel task execution using pools <img width="690" height="602" alt="Image" src="https://github.com/user-attachments/assets/4fc4424a-9d52-43d1-9060-b302d35eb4e0"; /> ### Use case/motivation _No response_ ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
