msumit commented on issue #36090:
URL: https://github.com/apache/airflow/issues/36090#issuecomment-2368378462
Sorry for the bummer folks, I don't think the solution I was thinking of is
generic enough, but here are 2 different approaches I could think of
- There is a cleanup function in BaseTrigger, which I think has not been
implemented by any other Trigger class. We can invoke that function when
calling cancel of the async task, but yeah that function needs to be
implemented in every Trigger class, very much like what @sunank200 did for a
couple of classes.
- We can send back the control to normal executors & call the `on_kill`
method of the operators. This would work for most of the cases, but there are a
couple of issues here as well
- Many operator's "on_kill" function isn't deferred aware, which means
it doesn't know that execution is being called from this function directly, and
it may be using some information coming out of the execute function.
- As we've to pass the task back to executors, the user would see
strange task state changes, like from success/failure to scheduled & then
running & then to the final state. Not sure if it's worth the effort to
introduce some transient states like marking_success or being_cancelled etc.
In short, we need an owner here who could either find a generic way or take
the lead in making all trigger classes `cancel` aware.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
