rajrao opened a new issue, #36539: URL: https://github.com/apache/airflow/issues/36539
### Description Currently when one pauses a DAG, it allows the currently executing task to complete and then leaves all downstream tasks to go to scheduled. I feel like there should be the ability to "Complete DAG run and then Pause", so that the entire DAG run can complete. ### Use case/motivation Often times we need to take down a certain service that a DAG is dependent on (eg: database server). Currently, we have the ability to pause the dag, which allows the executing task to complete and then puts all other downstream tasks in "scheduled" state and runs them once the Dag is unpaused. When the DAG is long running, we ideally would like to allow the entire DAG to complete so that the result of the DAG can be made available for customers and then pause the DAG. This way, I could put all these dags into "Complete and then Pause" mode and wait for all the work against my services to complete. After that I could take them offline, or do whatever work on them and then unpause all of the DAGs. Another use case is when we have to upgrade the Airflow. In such cases, again it would be nice to get all DAGs that are running to complete and then upgrade the Airflow. In the absence of such functionality, we are looking at adding a shortcircuit operator to our DAGs that would then check if a variable is set and if it is, then it would return False and short-circuit then run. This seems like adding an additional bit of plumbing that might be best implemented directly as part of DAG UI via an option like "Complete all downstream tasks and then Pause" For the Airflow upgrade use case, often times we have to take longer for the upgrade, as someone has to look at the DAG runs and as the complete, pause them and once they are all paused, then do the upgrade. And this needs someone to babysite Airflow. But, if there was a single option to allow ALL dags to "complete and then pause" this would be so much better. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
