Leondon9 commented on code in PR #73701:
URL: https://github.com/apache/airflow/pull/73701#discussion_r4371089611


##########
providers/amazon/src/airflow/providers/amazon/aws/operators/emr.py:
##########
@@ -1369,7 +1386,24 @@ def __init__(
         self.job_id: str | None = None
         self.deferrable = deferrable
         self.enable_application_ui_links = enable_application_ui_links
-        self.cancel_on_kill = cancel_on_kill
+        # In deferrable mode durable reconnects to the still-running job on 
clear, while cancel_on_kill
+        # (which only applies to the deferrable trigger) kills it: the two are 
mutually exclusive there.
+        # Because the worker re-executes before the trigger's on_kill runs, a 
durable retry would
+        # reconnect to a run that on_kill is about to cancel and then fail, so 
reject the contradiction
+        # and keep cancel_on_kill off when durable is enabled. Synchronous 
runs have no such race and
+        # cancel_on_kill does not apply to them, so the value is left 
untouched there.
+        if durable and deferrable:

Review Comment:
   One more thing worth flagging: since #72100, clearing a task discards its 
task state unless `keep_task_state` is set, so on 3.4+ `durable` reconnects on 
retries and on clears that keep task state, while a plain clear submits a new 
run. With `cancel_on_kill` off (the `durable` default) the old run then keeps 
going. I've documented that in the docstring rather than work around it, since 
it follows the "clearing starts over" behaviour, but let me know if you'd 
prefer a different default there.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to