maijh opened a new issue #12180: URL: https://github.com/apache/airflow/issues/12180
Apache Airflow version: v1109 executor: CeleryExecutor some time the scheduler create many sub scheduler process. When this happens, the CPU usage of these sub schduler process is at 99%, and then i can see web ui appear：**The scheduler does not appear to be running. Last heartbeat was received xxx ago.The DAGs list may not update, and new tasks will not be scheduled.** job_heartbeat_sec = 5 scheduler_heartbeat_sec = 5 run_duration = -1 num_runs = -1 processor_poll_interval = 1 min_file_process_interval = 30 here is some log i mark，but the scheduler log had no error etl 23905 12016 0 Oct17 ? 00:00:00 /bin/bash -c source /usr/soft/anaconda3/bin/activate etl_airflow_v1109 && airflow scheduler etl 23908 23905 1 Oct17 ? 03:13:04 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 23947 23908 1 Oct17 ? 02:40:24 airflow scheduler -- DagFileProcessorManager etl 30570 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30573 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30574 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30575 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30576 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30577 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30578 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30579 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30580 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30581 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler etl 30583 23908 99 03:07 ? 00:07:22 /root/etl_airflow_v1109/bin/python /root/etl_airflow_v1109/bin/airflow scheduler Please help. I would be happy to provide any other information needed ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
