[
https://issues.apache.org/jira/browse/AIRFLOW-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688469#comment-16688469
]
Andrew Harmon commented on AIRFLOW-2680:
----------------------------------------
I'm looking for a solution to the same issue. I don't think you always want the
SKIP status to propagate downstream. It would be nice to have some more control
of this
> Don't automatically percolate skipped state
> -------------------------------------------
>
> Key: AIRFLOW-2680
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2680
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: Andrei-Alin Popescu
> Assignee: Andrei-Alin Popescu
> Priority: Major
>
> Dear Airflow Maintainers,
>
> As part of our workflow, we have cases where all the upstream of a certain
> task A can be skipped. In this case, airflow seems to automatically mark A as
> skipped.
> However, this does not quite work for us, since there are changes external to
> the DAG which A needs to process, regardless of whether its upstream ran or
> not. Additionally, we require A to get into an "upstream_failed" state and
> not run if any its upstream tasks failed.
> I don't see a trigger rule to cover this, so what would be the best way to
> achieve this? I was thinking we could attach a DummyOperator as an upstream
> to A, which in a way marks the fact that A depends on some external data and
> needs to run anyway, but this can get really ugly for big DAGs.
> I was also thinking we could have a new trigger_rule, e.g. "no_failure",
> which would only trigger tasks if no upstream has failed. It differs from
> "all_success" in that it will also trigger if all upstream has been skipped,
> rather than percolating the skipped state on.
> I'd really appreciate your feedback on this, and I'd like to know if in fact
> there is already a good way of doing this with airflow that I don't know of.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
