Hi Vinod! That clarifies things for me. Another thing that I don't understand is whether Mesos still provides a fault-tolerant data store for frameworks. I recall reading about it in the Mesos paper, but it seems like now I'd need to do my own thing with Zookeeper or some other concurrency-safe data store. Is that the case now?
Thanks again, David On Tue, Apr 9, 2013 at 8:18 PM, Vinod Kone <[email protected]> wrote: > Hi David, > > The model of executors/tasks hasn't changed since we released 0.9.0. > That said, this is how things work. > > --> Framework registers with the master. > --> Master offers various slaves' resources to the framework. > --> Framework launches a task using the resources of an offer (an offer > maps uniquely to a slave). > --> When the slaves gets the task, it checks if the executor (see > TaskInfo) for that task is already running. > --> If yes, the task is sent to the executor and the executor driver calls > launchTask(). > --> If no, the slave launches the executor and then sends the task to > the executor. > --> You can launch multiple executors per slave as long as they have > different ExecutorIDs. > > --> How the tasks and executors interact is up to the executor > (framework) writers. > --> From Mesos point of view a task is something that consumes some > resources. > --> If you are using cgroups isolation on the slave, Mesos ensures that > the cgroup container running executor+tasks is properly sized (cpu, > memory etc). > > --> Another thing to note is that sending terminal status update > (e.g., TASK_FINISHED) for a finished task is crucial, for Mesos to properly > reoffer those resources to the framework and resize the container (if > using cgroups). > > --> Finally, for framework writers that don't want to write a > specialized executor, Mesos has a default executor called Command > Executor. All it does is wrap your command line in a shell script and > run it. > > --> Note that, a command executor can only run 1 task. It exits right > after the > shell command returns. > > We are in the process of revamping the documentation. So hopefully > these things will be more clear soon. > > HTH, > > > @vinodkone > > > On Tue, Apr 9, 2013 at 3:05 PM, David Greenberg <[email protected]> > wrote: > > I do not understand how tasks and executors work in the new mesos API. I > > assume that the model is still that a framework gets 1 scheduler, and > then > > that scheduler starts some number of executors. How do the executors and > > tasks interact, given that the scheduler only launches tasks? Are the > > executors implicitly created when needed, one per slave? > > > > Thanks! >
