Ash Berlin-Taylor created AIRFLOW-5686:
------------------------------------------
Summary: Deleting DAG can leave schedules/queued tasks consuming
pool slots
Key: AIRFLOW-5686
URL: https://issues.apache.org/jira/browse/AIRFLOW-5686
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Affects Versions: 1.10.5
Reporter: Ash Berlin-Taylor
Fix For: 1.10.7
If you delete a dag file when it had tasks in the scheduled or queued state,
those tasks instances are never touched again.
With the slight tweak in Pool (making the default pool an explicit one) this
now matters, and this ends up with the scheduler being "blocked" from running
new tasks wiht this message:
{noformat}
Figuring out tasks to run in Pool(name=default_pool) with -9022 open slots and
45 task instances ready to be queued {noformat}
The fix should be to set any task instance in a non-terminal state (None,
queued, scheduled, running, up_for_retry etc.) to "removed" inside
DAG.deactivate_stale_dags (which is already called on scheduler shutdown).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
