Currently the on_kill does not get triggered when you clear from the UI. As you mention, adding the 'fix' mentioned in the first comment of the issue, does no fix the problem as it does not trigger on the right operators context.
Im not sure what exact changes are planned for the next 1.10 release, but if the whole UI change is planned, this could be a good opportunity to also fix this bug. On Mon, Jan 8, 2018 at 2:55 PM, Ash Berlin-Taylor < [email protected]> wrote: > Without this change does on_kill ever get triggered? It seems like this > change is desired behaviour. > > As per the first comment https://issues.apache.org/ > jira/browse/AIRFLOW-1623?focusedCommentId=16171819& > page=com.atlassian.jira.plugin.system.issuetabpanels: > comment-tabpanel#comment-16171819 <https://issues.apache.org/ > jira/browse/AIRFLOW-1623?focusedCommentId=16171819& > page=com.atlassian.jira.plugin.system.issuetabpanels: > comment-tabpanel#comment-16171819> I'm not sure this is the right fix. It > also seems like this would end up running the on_kill in a different > process to the rest of the operator. > > I wonder if somewhere a signal handler is missing somewhere in one of the > `run --local` or `run --raw`. I tried to follow all the paths through from > ui to sig handlers but got stuck in a tiwsty maze of classes. (and was > attempting to do it just from reading the code)? > > > > On 8 Jan 2018, at 13:15, Driesprong, Fokko <[email protected]> wrote: > > > > Yes, for Spark this should work. Depending on the operator and the > > implementation: > > https://github.com/apache/incubator-airflow/blob/ > 3e6babe8ed8f8f281b67aa3f4e03bf3cfc1bcbaa/airflow/contrib/ > hooks/spark_submit_hook.py#L412-L428 > > > > However this is a big change in behaviour. I'm curious about the opinion > of > > others. > > > > Cheers, > > Fokko > > > > > > 2018-01-08 14:12 GMT+01:00 Milan van der Meer < > > [email protected]>: > > > >> Any help? :) > >> > >> On Thu, Dec 14, 2017 at 8:12 PM, Milan van der Meer < > >> [email protected]> wrote: > >> > >>> I recently openend the following PR: https://github.com/apache/ > >>> incubator-airflow/pull/2877 > >>> > >>> The problem is that on_kill is not called for operators when you clear > a > >>> task from the UI. > >>> Thats problematic when working with ex. spark clusters as the jobs on > the > >>> cluster need to be killed. > >>> > >>> The issue is in the core code of Airflow and Im not familiar enough > with > >>> the inner workings there. So I could use some directions on this one > from > >>> people who are familiar. > >>> > >>> For more info, check out the PR. > >>> > >>> Kind regards, > >>> Milan > >>> > >> > >> > >> > >> -- > >> > >> *Milan van der Meer* > >> > >> *Real**Impact* Analytics *| *Big Data Consultant > >> www.realimpactanalytics.com > >> > >> *BE *+32 498 45 96 22 <0032498459622>* | Skype *milan.vandermeer.ria > >> > > -- *Milan van der Meer* *Real**Impact* Analytics *| *Big Data Consultant www.realimpactanalytics.com *BE *+32 498 45 96 22 <0032498459622>* | Skype *milan.vandermeer.ria
