[
https://issues.apache.org/jira/browse/AIRFLOW-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor reassigned AIRFLOW-2684:
------------------------------------------
Assignee: (was: Ash Berlin-Taylor)
> scheduler_heartbeat_sec is not respected
> ----------------------------------------
>
> Key: AIRFLOW-2684
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2684
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.0
> Reporter: James Meickle
> Priority: Major
>
> In our staging environment, we leave Airflow debug logs on. We're generating
> immense amounts of logs because for both our 1.9 cluster and 1.10 cluster,
> scheduler_heartbeat_sec is not being respected. It's set to 5 in both
> environments, but produces logs like this every second:
>
> ```
> airflow_scheduler-stdout.log: [2018-06-27 13:47:20,272] \{{jobs.py:1685}}
> DEBUG - Ran scheduling loop in 0.00 seconds
> airflow_scheduler-stdout.log: [2018-06-27 13:47:20,273] \{{jobs.py:1686}}
> DEBUG - Sleeping for 1.00 seconds
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,273] \{{jobs.py:1607}}
> DEBUG - Starting Loop...
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,274] \{{jobs.py:1627}}
> INFO - Heartbeating the process manager
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,274]
> \{{dag_processing.py:523}} DEBUG - Queuing the following files for processing:
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,275] \{{jobs.py:1662}}
> INFO - Heartbeating the executor
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,275]
> \{{base_executor.py:103}} DEBUG - 0 running task instances
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,275]
> \{{base_executor.py:104}} DEBUG - 0 in queue
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,276]
> \{{base_executor.py:105}} DEBUG - 32 open slots
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,276]
> \{{base_executor.py:132}} DEBUG - Calling the <class
> 'airflow.executors.celery_executor.CeleryExecutor'> sync method
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,276]
> \{{celery_executor.py:80}} DEBUG - Inquiring about 0 celery task(s)
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,277] \{{jobs.py:1685}}
> DEBUG - Ran scheduling loop in 0.00 seconds
> airflow_scheduler-stdout.log: [2018-06-27 13:47:21,277] \{{jobs.py:1686}}
> DEBUG - Sleeping for 1.00 seconds
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
