Oliver Ricken created AIRFLOW-5188:
--------------------------------------
Summary: Dag Bag not in-sync with DB
Key: AIRFLOW-5188
URL: https://issues.apache.org/jira/browse/AIRFLOW-5188
Project: Apache Airflow
Issue Type: Bug
Components: DAG, database, scheduler, webserver
Affects Versions: 1.10.4
Environment: CentOS 7, Maria-DB, python 3.6.7, Airflow 1.10.4
Reporter: Oliver Ricken
Dear all,
we have upgraded our Airflow installation from 1.10.2 to 1.10.4 without any
immidiate hick-ups. However, now, we experience problems we did not observe
before the upgrade.
When changing the START_DATE of a DAG, we also change the DAG_ID to avoid any
misbehaviour of the scheduler due to changed stats of the DAG. This results in
new DAGs appearing in the Dag Bag and old ones disappearing – at least it did
before the upgrade.
Now, the picture is different. I changed the START_DATE and DAG_ID of a dag,
the Dag Bag lists it correctly (e.g. by manually issuing "airflow list_dags") –
only the latest version is displayed, the old one is gone.
However, the Web-UI still shows the old dag, along with the new dag. Removing
it via "airflow delete_dag" does not work, prompting "Dag is still in Dag Bag,
remove dag file first.". However, accessing dag details via the Web-UI fails,
prompting "dag is missing", indicating that is NOT in the Dag Bag anymore
(which seems to be correct).
To complete the picture: both dags are present in the DB.
Restarting the webserver did not help either. Restarting the scheduler has not
yet been an option due to running dags. This would be my next escalation step.
Can anyone comment on similar observations particularly with respect to having
upgraded to 1.10.4?
Cheers
Oliver
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
