> > I have the status of TASK_COMPLETED being sent via the driver, followed by > a wait of about 5 secs
This is needed because of https://issues.apache.org/jira/browse/MESOS-243. A 1 second sleep should be ample. > However, I'd still like to hear any thoughts on the approach of using one > task per executor to simplify my executor code by not having the forking > and process management. Plus, the benefit of resource isolation now > applying to tasks. Especially if there's reasons not to do this. This is perfectly fine and is what is done by several general purpose frameworks, such as Aurora. There's no reason *not* to do this, especially if your tasks are independent and are arbitrary forked processes. You're correct that task-level isolation is what you'll want, in this case. Other frameworks, such as Spark, may have different semantics for a Task. A Task may simply be an operation performed in a thread inside the Executor, rather than an arbitrary forked process. The semantics of a Task are entirely up to you. If you don't want to implement an Executor and your Task merely consists of forking an arbitrary process, you can use the built-in "Command Executor". You can launch a task directly in this manner by specifying a CommandInfo inside your TaskInfo (see the documentation in mesos.proto). Unless you're using the "Command Executor", you will still need to implement "forking and process management". Also, I assume you mean TASK_FINISHED instead of TASK_COMPLETED? On Mon, Mar 10, 2014 at 4:39 PM, Sharma Podila <spod...@netflix.com> wrote: > I will have to take back the question on why I am getting a TASK_LOST; I > had a bug that prevented the TASK_COMPLETED to be sent out before exiting. > I noticed it only after sending my email out! > > However, I'd still like to hear any thoughts on the approach of using one > task per executor to simplify my executor code by not having the forking > and process management. Plus, the benefit of resource isolation now > applying to tasks. Especially if there's reasons not to do this. > > Thanks. > > Sharma > > > On Mon, Mar 10, 2014 at 4:09 PM, Sharma Podila <spod...@netflix.com>wrote: > >> I have a question on the Executors. I see that a framework can use 1 or >> more executors on a slave. At finest granularity, one could launch each >> task in a separate executor (our tasks are relatively long lived). This at >> least seems like a good choice since resource isolation of executors now >> happens for each task. And, I could eliminate the work in my executor of >> forking processes to execute tasks. >> >> When a task completes (either normally or by getting a kill request), I >> would like the executor to exit as well. Playing with this, I have the >> status of TASK_COMPLETED being sent via the driver, followed by a wait of >> about 5 secs (not sure I need this for any mesos communication to happen >> for status), and then, say, a System.exit(). However, this is producing a >> TASK_LOST to my framework scheduler. Shouldn't the previous sending of >> TASK_COMPLETED status prevent the TASK_LOST update? >> >> I am using Mesos 0.16. >> >> Sharma >> >> >
