Alan Ma created AIRFLOW-2113:
--------------------------------
Summary: DagRuns sometimes does not execute callbacks
Key: AIRFLOW-2113
URL: https://issues.apache.org/jira/browse/AIRFLOW-2113
Project: Apache Airflow
Issue Type: Bug
Reporter: Alan Ma
Assignee: Alan Ma
This originally arose from the missing notification from the on_failure and
on_success callback at the dag level. The stack trace is as follows:
{code:java}
[2018-02-07 07:00:08,145] \{models.py:2984} DagFileProcessor172 INFO -
Executing dag callback function:
<pandora_airflow_plugin.hooks.general_notification_hook
.GeneralNotifyFailed instance at 0x7fec9d8ad368>
[2018-02-07 07:00:08,161] \{models.py:168} DagFileProcessor172 INFO - Filling
up the DagBag from /home/charon/.virtualenvs/airflow/airflow_home/dags
Dag: <DAG: hive_migration_dag>, paused: False
Dag: <DAG: continuously_refresh_ui>, paused: False
Dag: <DAG: consul_ttl_check>, paused: False
Dag: <DAG: example_analytics_dag>, paused: False
Dag: <DAG: postgres_vacuum>, paused: False
[2018-02-07 07:00:12,103] \{jobs.py:354} DagFileProcessor172 ERROR - Got an
exception! Propagating...
Traceback (most recent call last):
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/jobs.py",
line 346, in helper
pickle_dags)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/utils/db.py",
line 53, in wrapper
result = func(*args, **kwargs)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/jobs.py",
line 1586, in process_file
self._process_dags(dagbag, dags, ti_keys_to_schedule)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/jobs.py",
line 1175, in _process_dags
dag_run = self.create_dag_run(dag)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/utils/db.py",
line 53, in wrapper
result = func(*args, **kwargs)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/jobs.py",
line 747, in create_dag_run
dag.handle_callback(dr, success=False, reason='dagrun_timeout', session=session)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/utils/db.py",
line 53, in wrapper
result = func(*args, **kwargs)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/airflow/models.py",
line 2990, in handle_callback
d = dagrun.dag or DagBag().get_dag(dag_id=dagrun.dag_id)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py",
line 237, in __get__
return self.impl.get(instance_state(instance), dict_)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py",
line 579, in get
value = state._load_expired(state, passive)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/sqlalchemy/orm/state.py",
line 592, in _load_expired
self.manager.deferred_scalar_loader(self, toload)
File
"/home/charon/.virtualenvs/airflow/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py",
line 644, in load_scalar_attributes
(state_str(state)))
DetachedInstanceError: Instance <DagRun at 0x7fec9d8c8e90> is not bound to a
Session; attribute refresh operation cannot proceed
[2018-02-07 07:00:31,003] \{jobs.py:343} DagFileProcessor208 INFO - Started
process (PID=7813) to work on
/home/charon/.virtualenvs/airflow/airflow_home/dags/c
haron-airflow/dags/inapp_vendor_sku_breakdown.py\{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
