John Smodic created AIRFLOW-3356:
------------------------------------
Summary: Scheduler gets stuck for certain DAGs
Key: AIRFLOW-3356
URL: https://issues.apache.org/jira/browse/AIRFLOW-3356
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Affects Versions: 1.9.0
Reporter: John Smodic
I observe the scheduler getting stuck for certain DAGs:
Nov 15 19:11:48 ip-172-16-13-120 python3.6[1319]: File Path PID Runtime Last
Runtime Last Run
Nov 15 19:11:48 ip-172-16-13-120 python3.6[1319]:
/home/ubuntu/airflow/dags/stuck_dag.py 14241 19977.55s 1.05s 2018-11-15T13:38:47
Nov 15 19:11:48 ip-172-16-13-120 python3.6[1319]:
/home/ubuntu/airflow/dags/not_stuck_dag.py 19906 0.05s 1.05s 2018-11-15T19:11:44
The "Runtime" of the stuck DAG's scheduling process is huge and I can't tell
what it's doing. There's no mention of that DAG in the scheduler logs otherwise.
The mapped process looks like this:
ubuntu 14241 0.0 0.3 371132 63232 ? S 13:38 0:00 /usr/bin/python3.6
/usr/local/bin/airflow scheduler
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
