[jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930940#comment-15930940 ] Dan Davydov commented on AIRFLOW-992: - Good point I'll do both edit + comment in the future > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930937#comment-15930937 ] Bolke de Bruin commented on AIRFLOW-992: That would make sense I think. Have you tested it yet that way? On a side note: Would you mind providing your edits as comments? It is difficult to find out what has changed if reading from a phone. > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927332#comment-15927332 ] Dan Davydov commented on AIRFLOW-992: - This is the expected behavior after the semantics changes made here: https://github.com/apache/incubator-airflow/pull/2125 and later documented/clarified here: https://github.com/apache/incubator-airflow/pull/2151/files > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927323#comment-15927323 ] Bolke de Bruin commented on AIRFLOW-992: Do you have anything to reproduce it with? Preferably a test or a dag > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > Actual: > A gets run > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
