It's the responsibility of each executor to shut itself down when it decides its no longer needed. The idea here is that an executor can stay running with zero tasks, so that when the scheduler decides to launch a task, the scheduler's already initialized, thus saving startup time.
To avoid the problem you're seeing, have killTask schedule the sys.exit(0) to occur in 2 seconds--this will give time for your driver.sendStatusUpdate(taskid, TASK_KILLED) to fully send the status, and then you can safely exit the executor. On Thu, Jul 23, 2015 at 8:10 AM Eren Güven <[email protected]> wrote: > Hello, > > *TL;DR*: What is the proper way to kill executor process(tree) from > within executor code? > > I have an executor written in python that I use with Marathon to launch my > tasks. I use it to have a control on task status switch STAGING => RUNNING. > I launch a long-running subprocess from the `launchTask` > > My problem is that (following the examples such as RENDLER) if I do > `sys.exit(0)` at the end of `killTask`, I get "Failed to call executor's > killTask" on stderr and get TASK_LOST. If I omit the `sys.exit(0)` then a > leftover executor process stays on the slave. > > Cheers > Eren >
