Thanks all. Jens - I mentioned it above:
> If you are someone building or maintaining a trigger that manages external work, you can consider adopting to this hook and implementing it for your trigger class, Airflow 3.3 will launch this. Jason - yep, I can imagine that pain :) Thanks & Regards, Amogh Desai On Fri, Apr 24, 2026 at 3:23 AM Zhe-You(Jason) Liu <[email protected]> wrote: > Thanks Amogh for adding the `on_kill` method to `BaseTrigger`! > > Without your change, the only way for me to achieve an explicit kill from > the CLI or UI is by adding a `@teardown` task at the end of the external > job submission for the "sentinel" purpose. > Your change even makes this better, we can achieve the explicit kill with > an atomic task! > > Best, > Jason > > On Fri, Apr 24, 2026 at 4:14 AM Jens Scheffler <[email protected]> > wrote: > > > Thanks Amogh for adding this! > > > > In deed this was missing for a long time! > > > > You did not tell about but we can assume plan is to release this with > > Airflow 3.3.0? > > > > On 23.04.26 18:03, Aritra Basu wrote: > > > This is quite useful! Nice! > > > Thanks and Regards, > > > Aritra Basu > > > > > > > > > On Thu, Apr 23, 2026 at 9:17 PM Amogh Desai <[email protected]> > > wrote: > > > > > >> Hi Again, > > >> > > >> Created an issue tracking the migration of provider triggers from the > > old > > >> CancelledError > > >> pattern to the new on_kill() hook: > > >> https://github.com/apache/airflow/issues/65733 > > >> > > >> Three triggers need updating — Dataproc, EMR Serverless, and > Kubernetes. > > >> Each is a > > >> contained change in a single file. Good first contribution if you want > > to > > >> get familiar with the > > >> triggerer. > > >> > > >> Drop a comment on the issue if you want to pick one up. > > >> > > >> Thanks & Regards, > > >> Amogh Desai > > >> > > >> > > >> On Thu, Apr 23, 2026 at 7:40 PM Kaxil Naik <[email protected]> > wrote: > > >> > > >>> Finally, this has been troubling users since Triggerer was added. > > >>> > > >>> On Thu, 23 Apr 2026 at 14:59, Elad Kalif <[email protected]> wrote: > > >>> > > >>>> This is great! well done! > > >>>> > > >>>> On Thu, Apr 23, 2026 at 4:42 PM Amogh Desai <[email protected]> > > >>> wrote: > > >>>>> Hey All, > > >>>>> > > >>>>> Wanted to call out a new hook we added to BaseTrigger — > `on_kill()`, > > >>> PR: > > >>>>> https://github.com/apache/airflow/pull/65590. > > >>>>> > > >>>>> *Why does it exist?* > > >>>>> > > >>>>> If your trigger polls an external job (BigQuery, Databricks, > > >> Dataproc, > > >>>>> etc.), there was no clean way to cancel that job when > > >>>>> a user kills the deferred task, either by marking it as success, > > >>> failure, > > >>>>> or clearing it. `cleanup()` wasn't the right place — > > >>>>> it fires on every trigger exit including triggerer restarts and > > >> rolling > > >>>>> deploys, so putting cancellation there would kill in-flight > > >>>>> work on every redeploy. > > >>>>> > > >>>>> `on_kill()` solves this. It only fires when a user explicitly acts > on > > >>> the > > >>>>> task. Not on restart, not on redistribution. > > >>>>> > > >>>>> *How it works* > > >>>>> > > >>>>> The triggerer passes a private sentinel via > > >> `asyncio.Task.cancel(msg)` > > >>> to > > >>>>> distinguish user kills from other exits — no DB > > >>>>> round-trip needed. There's a configurable timeout (*[triggerer] > > >>>>> on_kill_timeout, default 30s*) so a slow external API call > > >>>>> cannot block the triggerer. > > >>>>> > > >>>>> Databricks is updated as a reference: > > >>>>> https://github.com/apache/airflow/pull/65672 > > >>>>> > > >>>>> Both `DatabricksExecutionTrigger` and > > >>>>> `DatabricksSQLStatementExecutionTrigger` now implement it. > > >>>>> > > >>>>> *Who benefits from this?* > > >>>>> > > >>>>> If you are someone building or maintaining a trigger that manages > > >>>> external > > >>>>> work, you can consider adopting to this hook > > >>>>> and implementing it for your trigger class, Airflow 3.3 will launch > > >>> this. > > >>>>> No compat guard is needed, Airflow < 3.2 the method just never gets > > >>>> called. > > >>>>> *Next steps* > > >>>>> > > >>>>> I pulled some data from the repo and these are candidates for this > > >>>>> migration. > > >>>>> > > >>>>> Google Provider: DataprocSubmitTrigger and > > >>> DataprocSubmitJobDirectTrigger > > >>>>> in the google provider. Currently catches > > >>>>> CancelledError in run() and calls hook.cancel_job(job_id) after > > >>> checking > > >>>>> safe_to_cancel(). > > >>>>> > > >>>>> Amazon Provider: EmrServerlessStartJobTrigger. Same pattern. > > >>>> CancelledError > > >>>>> handler with safe_to_cancel() calling > > >>>>> hook.conn.cancel_job_run(). Straightforward migration. > > >>>>> > > >>>>> CNCF provider: KubernetesPodTrigger in cncf/kubernetes. Has an > > >>>>> on_kill_action parameter controlling whether to > > >>>>> delete/patch the pod. The cancellation logic lives in a > > >> CancelledError > > >>>>> handler. Move it to on_kill(). > > >>>>> > > >>>>> Happy to help with reviews for these. > > >>>>> > > >>>>> > > >>>>> Thanks & Regards, > > >>>>> Amogh Desai > > >>>>> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
