josh-fell commented on a change in pull request #17421:
URL: https://github.com/apache/airflow/pull/17421#discussion_r758449147
##########
File path: airflow/operators/python.py
##########
@@ -185,16 +186,37 @@ def execute(self, context: Dict):
class ShortCircuitOperator(PythonOperator, SkipMixin):
"""
Allows a workflow to continue only if a condition is met. Otherwise, the
- workflow "short-circuits" and downstream tasks are skipped.
+ workflow "short-circuits" and downstream tasks are skipped. The
short-circuiting can be configured to
+ operated in one of two modes: "hard" or "soft". In "hard" mode, the
default mode, all downstream tasks
+ are skipped without considering the ``trigger_rule`` defined for tasks.
In "soft" mode, the
+ ``trigger_rule`` is respected.
The ShortCircuitOperator is derived from the PythonOperator. It evaluates a
- condition and short-circuits the workflow if the condition is False. Any
- downstream tasks are marked with a state of "skipped". If the condition is
- True, downstream tasks proceed as normal.
+ condition and short-circuits the workflow if the condition is False. Any
downstream tasks are marked with
+ a state of "skipped" based on the short-circuiting mode configured. If the
condition is True, downstream
+ tasks proceed as normal.
- The condition is determined by the result of `python_callable`.
+ The condition is determined by the result of ``python_callable``.
+
+ :param mode: If set to "hard", all downstream tasks from this operator
task will be skipped. This is
+ the default behavior. If set to "soft", downstream tasks will be
skipped but the ``trigger_rule``
+ defined for a task will be respected.
+ :type mode: bool
"""
+ modes = ["hard", "soft"]
Review comment:
In previous iterations we did have "soft/hard modes" and `do_hard_short`
but the question was what do they _mean_? While something like
`ignore_downstream_trigger_rules` is long, it is pretty explicit and more
understandable than the `modes`/`do_hard_short` names. IMO the same goes with
`soft_skip` (or `soft_fail` in the sensors). They are succinct parameter names
but what do they _mean_? Also, there is more consistency (albeit loosely)
around `ignore_...` flag nomenclature in Airflow.
Even if users look at the documentation/docstrings anyway to figure out what
the parameter does, I still think following the "explicit is better than
implicit" principle is the way to go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
