peschue commented on issue #13198: URL: https://github.com/apache/airflow/issues/13198#issuecomment-804938134
I still have this problem and I run version 2.0.1. Maybe it is because of my special DAG setup: I have a DAG that runs each day, and an ExternalTaskSensor that waits for the previous day's completion and an External TaskSensor that waits for the previous week's completion (part of the DAG is skipped and runs only once a week). If I clear many of the DAGs with the intention to let it catch up in the correct order (by waiting until one week is done and then doing the next week) I get flooded with "removed" task states from everything depending on the task sensors and the log says "Already has 18 active runs, not queuing any tasks for run" (although max_concurrency is 100) and clearing the removed tasks just makes them re-appear. If I deactivate the DAG, I can clear the "removed" tasks and they appear only if I reactivate the DAG. Still, nothing gets scheduled so the task sensors have no chance to ever succeed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
