josh-fell opened a new pull request #17421: URL: https://github.com/apache/airflow/pull/17421
Closes: #7858 Adding the ability to configure the `ShortCircuitOperator` to respect trigger rules for downstream tasks. Currently this operator ignores all trigger rules and forcibly skips all downstream tasks. However, there are use cases in which downstream tasks from the `ShortCircuitOperator` have trigger rules applied such that said tasks should execute even if upstream tasks are skipped by the operator (e.g. multiple branches that execute in parallel, one branch can be short-circuited at some point, and the branches converge). This PR adds a new boolean parameter, `do_hard_short`, which allows users to have the `ShortCircuitOperator` perform a "hard short" (i.e. blindly skip all downstream tasks; the current behavior) or a "soft short" (i.e. the immediate, downstream task(s) are skipped only and the Scheduler is left to handle the trigger rules appropriately). The unit tests for the `ShortCircuitOperator` were refactored as part of this change as well. They were emitting false-positives and could be more succinct as they was some clear copy/paste from the `BranchPythonOperator` tests. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
