In case my previous email didn't reach you correctly or went to spam, --------- Yes, Mesos doesn't provide a fault-tolerant data store for frameworks out of the box.
Having said that, Mesos does provide an abstraction called 'State' ( see src/state/) that framework writers can use to provide reliability guarantees. You can see how AirBnB implemented their Chronos<https://github.com/airbnb/chronos> framework (on top of Mesos) to get an idea on how to use the State abstraction. Cheers, -- Vinod On Thu, Apr 11, 2013 at 6:37 PM, David Greenberg <[email protected]>wrote: > Hi Vinod! That clarifies things for me. > > Another thing that I don't understand is whether Mesos still provides a > fault-tolerant data store for frameworks. I recall reading about it in the > Mesos paper, but it seems like now I'd need to do my own thing with > Zookeeper or some other concurrency-safe data store. Is that the case now? > > Thanks again, > David > > > On Tue, Apr 9, 2013 at 8:18 PM, Vinod Kone <[email protected]> wrote: > > > Hi David, > > > > The model of executors/tasks hasn't changed since we released 0.9.0. > > That said, this is how things work. > > > > --> Framework registers with the master. > > --> Master offers various slaves' resources to the framework. > > --> Framework launches a task using the resources of an offer (an offer > > maps uniquely to a slave). > > --> When the slaves gets the task, it checks if the executor (see > > TaskInfo) for that task is already running. > > --> If yes, the task is sent to the executor and the executor driver > calls > > launchTask(). > > --> If no, the slave launches the executor and then sends the task to > > the executor. > > --> You can launch multiple executors per slave as long as they have > > different ExecutorIDs. > > > > --> How the tasks and executors interact is up to the executor > > (framework) writers. > > --> From Mesos point of view a task is something that consumes some > > resources. > > --> If you are using cgroups isolation on the slave, Mesos ensures that > > the cgroup container running executor+tasks is properly sized (cpu, > > memory etc). > > > > --> Another thing to note is that sending terminal status update > > (e.g., TASK_FINISHED) for a finished task is crucial, for Mesos to > properly > > reoffer those resources to the framework and resize the container (if > > using cgroups). > > > > --> Finally, for framework writers that don't want to write a > > specialized executor, Mesos has a default executor called Command > > Executor. All it does is wrap your command line in a shell script and > > run it. > > > > --> Note that, a command executor can only run 1 task. It exits right > > after the > > shell command returns. > > > > We are in the process of revamping the documentation. So hopefully > > these things will be more clear soon. > > > > HTH, > > > > > > @vinodkone > > > > > > On Tue, Apr 9, 2013 at 3:05 PM, David Greenberg <[email protected]> > > wrote: > > > I do not understand how tasks and executors work in the new mesos API. > I > > > assume that the model is still that a framework gets 1 scheduler, and > > then > > > that scheduler starts some number of executors. How do the executors > and > > > tasks interact, given that the scheduler only launches tasks? Are the > > > executors implicitly created when needed, one per slave? > > > > > > Thanks! > > >
