[
https://issues.apache.org/jira/browse/AIRFLOW-316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-316:
--------------------------------
External issue URL: https://github.com/apache/incubator-airflow/pull/1654
> Always check DB state for Backfill Job execution
> ------------------------------------------------
>
> Key: AIRFLOW-316
> URL: https://issues.apache.org/jira/browse/AIRFLOW-316
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Dan Davydov
> Assignee: Dan Davydov
> Labels: backfill
>
> Always check DB state and not just the local state for backfill jobs for
> determining which task instances have not yet completed execution. This is to
> avoid potential race conditions with e.g. two backfill jobs running the same
> task instance.
> At Airbnb we have noticed a race condition with the subdag operator (which
> uses the backfill job) where all of the tasks in the subdag complete
> successfully, but the parent DAG is still waiting on a task in the subdag to
> succeed. Resolving this JIRA should fix the subdag operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
