[
https://issues.apache.org/jira/browse/AIRFLOW-4480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huihua Zhang updated AIRFLOW-4480:
----------------------------------
Description:
AIRFLOW-511 introduced on_success_callback and on_failure_callback callbacks
and passed test_dagrun_success_callback and test_dagrun_failure_callback tests.
This is great. But for dagrun_timeout, airflow hasn't covered the case.
Although there is one line to handle_callback when dag exceeded dagrun_timeout,
actually it won't ever trigger any callbacks. The reason is the dag
[here|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L802] has
no on_success_callback and on_failure_callback which are both None. These DAGs
are from
[DagFileProcessor|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L307]
which parsed python files to get DAG ids and then get DAGs with None callbacks.
I've checked airflow database, seems it doesn't store callbacks in the
database. Are there any approaches that we can get DAGs with callbacks along
with other DAG settings in
[SchedulerJob|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L399]?
Or in brief how to trigger callbacks when dagrun_timeout? Thanks.
was:
AIRFLOW-511 introduced on_success_callback and on_failure_callback callbacks
and passed test_dagrun_success_callback and test_dagrun_failure_callback tests.
This is great. But for dagrun_timeout, airflow hasn't covered the case.
Although there is one line to handle_callback when dag exceeded dagrun_timeout,
actually it won't ever trigger any callbacks. The reason is the dag
[here|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L790] has
no on_success_callback and on_failure_callback which are both None. These DAGs
are from
[DagFileProcessor|https://github.com/apache/airflow/blob/master/airflow/jobs.py#L300]
which parsed python files to get DAG ids and then get DAGs with None callbacks.
I've checked airflow database, seems it doesn't store callbacks in the
database. Are there any approaches that we can get DAGs with callbacks along
with other DAG settings in
[SchedulerJob|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L393]?
Or in brief how to trigger callbacks when dagrun_timeout? Thanks.
> dagrun_timeout won't trigger callbacks
> --------------------------------------
>
> Key: AIRFLOW-4480
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4480
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.3
> Reporter: Huihua Zhang
> Assignee: Ace Haidrey
> Priority: Minor
>
> AIRFLOW-511 introduced on_success_callback and on_failure_callback callbacks
> and passed test_dagrun_success_callback and test_dagrun_failure_callback
> tests. This is great. But for dagrun_timeout, airflow hasn't covered the
> case. Although there is one line to handle_callback when dag exceeded
> dagrun_timeout, actually it won't ever trigger any callbacks. The reason is
> the dag
> [here|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L802] has
> no on_success_callback and on_failure_callback which are both None. These
> DAGs are from
> [DagFileProcessor|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L307]
> which parsed python files to get DAG ids and then get DAGs with None
> callbacks.
> I've checked airflow database, seems it doesn't store callbacks in the
> database. Are there any approaches that we can get DAGs with callbacks along
> with other DAG settings in
> [SchedulerJob|https://github.com/apache/airflow/blob/1.10.3/airflow/jobs.py#L399]?
> Or in brief how to trigger callbacks when dagrun_timeout? Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
