gvazharov opened a new issue #15369:
URL: https://github.com/apache/airflow/issues/15369
Hello folks,
I am using Airflow 2.0 with the KubernetesExecutor and Postgres DB as config
database, after more than a month in production one of my dags has stopped
working out of no where.
The problem is that the scheduler is creating new dag runs, but the tasks
inside the dag are not queued, in the scheduler logs could be found the
following:
_[2021-04-14 14:51:02,666] {{scheduler_job.py:1667}} INFO - DAG
tool_statistic_dag already has 16 active runs, not queuing any tasks for run
2021-04-14 12:20:00+00:00_
In the UI all previous dag runs are successfully finished and no running
tasks are displayed, I've checked also the status of all my previous dag runs
in Postgres, but didn't found anything that is running and could cause any
vulnerability.
I am not quite sure how to reproduce this behavior, but I had the same
problem in my test environment and the only solution to fix the dag was to
delete it and import it again, after that everything start to run smoothly
again. Because now the problem is in the production environment I don't want to
do that again, therefore I will be glad to get your support.
Just for info, this is my first Dag, that is using Task Groups.
Here you can see my tree view:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
