[ https://issues.apache.org/jira/browse/AIRFLOW-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bolke de Bruin resolved AIRFLOW-812. ------------------------------------ Resolution: Fixed Fix Version/s: 1.8.1 Issue resolved by pull request #2027 [https://github.com/apache/incubator-airflow/pull/2027] > Scheduler job terminates when there is no dag file despite run_numbers and > run_duration set to unlimited. > --------------------------------------------------------------------------------------------------------- > > Key: AIRFLOW-812 > URL: https://issues.apache.org/jira/browse/AIRFLOW-812 > Project: Apache Airflow > Issue Type: Bug > Components: utils > Reporter: Feng Lu > Assignee: Feng Lu > Fix For: 1.8.1 > > > When scheduler run_numbers and run_duration are set to -1 (unlimited), we > expect the scheduler to run continuously. However, when dag folder is empty, > the scheduler terminates immediately. This is due to a bug in > utils/dag_processing when checking max_runs_reached. > Details: > airflow scheduler > [2017-01-27 08:20:50,433] {__init__.py:57} INFO - Using executor > SequentialExecutor > [2017-01-27 08:20:50,508] {driver.py:120} INFO - Generating grammar tables > from /usr/lib/python2.7/lib2to3/Grammar.txt > [2017-01-27 08:20:50,529] {driver.py:120} INFO - Generating grammar tables > from /usr/lib/python2.7/lib2to3/PatternGrammar.txt > ____________ _____________ > ____ |__( )_________ __/__ /________ __ > ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / / > ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ / > _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/ > [2017-01-27 08:20:50,771] {jobs.py:514} ERROR - Cannot use more than 1 thread > when using sqlite. Setting max_threads to 1 > [2017-01-27 08:20:50,786] {jobs.py:1222} INFO - Starting the scheduler > [2017-01-27 08:20:50,787] {jobs.py:1238} INFO - Processing files using up to > 1 processes at a time > [2017-01-27 08:20:50,787] {jobs.py:1240} INFO - Running execute loop for -1 > seconds > [2017-01-27 08:20:50,787] {jobs.py:1242} INFO - Processing each file at most > None times > [2017-01-27 08:20:50,787] {jobs.py:1244} INFO - Process each file at most > once every 0 seconds > [2017-01-27 08:20:50,787] {jobs.py:1246} INFO - Checking for new files in > /home/fenglu/airflow/dags every 300 seconds > [2017-01-27 08:20:50,787] {jobs.py:1249} INFO - Searching for files in > /home/fenglu/airflow/dags > [2017-01-27 08:20:50,787] {jobs.py:1252} INFO - There are 0 files in > /home/fenglu/airflow/dags > [2017-01-27 08:20:50,787] {jobs.py:1314} INFO - Resetting state for orphaned > tasks > [2017-01-27 08:20:50,789] {jobs.py:1326} INFO - Removing old import errors > [2017-01-27 08:20:50,795] {jobs.py:1363} INFO - Heartbeating the process > manager > [2017-01-27 08:20:50,796] {jobs.py:1400} INFO - Heartbeating the executor > [2017-01-27 08:20:51,797] {jobs.py:1432} INFO - Exiting loop as all files > have been processed None times > [2017-01-27 08:20:51,797] {jobs.py:1448} INFO - Deactivating DAGs that > haven't been touched since 2017-01-27T08:20:50.795782 > [2017-01-27 08:20:51,801] {jobs.py:1271} INFO - Exited execute loop -- This message was sent by Atlassian JIRA (v6.3.4#6332)