[
https://issues.apache.org/jira/browse/AIRFLOW-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365171#comment-15365171
]
ASF subversion and git services commented on AIRFLOW-316:
---------------------------------------------------------
Commit 635c97a60636c223d374452258465fdc337f0913 in incubator-airflow's branch
refs/heads/master from [~aoen]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=635c97a ]
[AIRFLOW-316] Always check DB state for Backfill Job execution
Closes #1654 from aoen/ddavydov/dont_skip_db_state_check_for_subdag
Always check DB state and not just the local state for backfill jobs for
determining which task instances have not yet completed execution.
This is to avoid potential race conditions with e.g. two backfill jobs
running the same task instance.
> Always check DB state for Backfill Job execution
> ------------------------------------------------
>
> Key: AIRFLOW-316
> URL: https://issues.apache.org/jira/browse/AIRFLOW-316
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Dan Davydov
> Assignee: Dan Davydov
> Labels: backfill
>
> Always check DB state and not just the local state for backfill jobs for
> determining which task instances have not yet completed execution. This is to
> avoid potential race conditions with e.g. two backfill jobs running the same
> task instance.
> At Airbnb we have noticed a race condition with the subdag operator (which
> uses the backfill job) where all of the tasks in the subdag complete
> successfully, but the parent DAG is still waiting on a task in the subdag to
> succeed. Resolving this JIRA should fix the subdag operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
