Christopher Groskopf created AIRFLOW-1738:
---------------------------------------------
Summary: Deleting dag.py after manual run causes Airflow to hang
on next manual run
Key: AIRFLOW-1738
URL: https://issues.apache.org/jira/browse/AIRFLOW-1738
Project: Apache Airflow
Issue Type: Bug
Components: DagRun
Environment: Airflow 1.8.2, running on Alpine Docker images with
Celery.
Reporter: Christopher Groskopf
Priority: Minor
We're building integration tests around Airflow that upload a new {{dag.py}}
file, trigger it's execution (using
{{airflow.api.common.experimental.trigger_dag}}), wait for it to complete, and
then cleanup. However, we've run into a peculiar set of circumstances where if
we do this multiple times in sequence, Airflow will sometime hang indefinitely
when we try to trigger our next dag.
I've traced this and determined that the issue only happens if remove the dag
files after each test. I speculate that this is because triggering a dag
invokes {{create_dagrun}} which in turn invokes {{verify_integrity}}, which I
believe may be choking on the sudden disappearance of the DAG files. I'm
certain this is correct, but it's the best explanation I've been able to come
up with.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
