grayver commented on issue #11331:
URL: https://github.com/apache/airflow/issues/11331#issuecomment-707878281
It seems to be I figured out what the problem is. We have Airflow scheduler
installed as a daemon service on Ubuntu machine. Service command is following:
```
/bin/bash -c 'source /home/airflow/airflow_venv/bin/activate &&
/home/airflow/airflow scheduler -n ${SCHEDULER_RUNS} --pid
/run/airflow/scheduler.pid'
```
SCHEDULER_RUNS is set to 5 in environment variables. So, scheduler starts,
makes 5 loops of scanning DAG files and then stops. Linux daemon restart policy
automatically restart it again. However, if long-running task is still running
after service restart (it's not stopped) scheduler doesn't grab new tasks until
it finishes.
Setting SCHEDULER_RUNS to -1 solved the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
