hussein-awala commented on issue #30491: URL: https://github.com/apache/airflow/issues/30491#issuecomment-1498234484
First of all, this is not a bug, rather it is a new feature request. As long as the dag run has at least one unfinished task(state not in [success, failed, skipped, upstream_failed, removed]), and the dag run is not timed out and not manually cleared , its state should remain running. For the new proposed state `stalled`, I think this can add a lot of complexity to the scheduler job, which should process the dag runs with the state `stalled` in the critical section like the state `running` to check if the tasks are unblocked or they still somehow blocked, and should not run any task in the stalled dag runs before passing them to running, respecting that the count of running tasks is less than or equal to the max active dag runs. IMHO this can be a good feature in some use cases, especially in large dags that have a small number of tasks with depend_on_past or with a small pool size. I convert it to discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
