dstandish commented on code in PR #30270:
URL: https://github.com/apache/airflow/pull/30270#discussion_r1147242111
##########
airflow/ti_deps/deps/trigger_rule_dep.py:
##########
@@ -263,8 +280,28 @@ def _iter_upstream_conditions() ->
Iterator[ColumnOperators]:
elif trigger_rule == TR.ALL_SKIPPED:
if success or failed:
new_state = TaskInstanceState.SKIPPED
-
+ elif trigger_rule == TR.ALL_DONE_SETUP_SUCCESS:
+ # when there is an upstream setup and they have all skipped,
then skip
+ if upstream_done and upstream_setup and skipped_setup >=
upstream_setup:
+ new_state = TaskInstanceState.SKIPPED
+ logging.warning(
+ "ti=%s skipped_setup >= upstream_setup (%s >= %s),
marking skipped",
+ ti.task_id,
+ skipped_setup,
+ upstream_setup,
+ )
+ elif upstream_done and upstream_setup > success_setup:
+ # when there is an upstream setup, they must all succeed
+ # otherwise, behave same as all done.
Review Comment:
Oh but so you're saying that new trigger rule was not specified in proposal.
That's correct. I guess I think of it as implementation detail but, of course
such details matter. The AIP made general predictions about how implementation
would proceed, thinking that not setting up deps was going to be better but we
found that route problematic. There's some ongoing discussion of course on the
dev list concerning some proposed changes but that's more about API than
implementation. Do you have concerns with adding a new trigger rule to support
this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
