Re: Service Scheduling in Mesos
From what I understand, the Omega paper was written in 2012. It's great. Much has been added to Apache Mesos since. Particularly w.r.t. scheduling services. Also, the two-level categorization arguably has evolved further. On Thu, Sep 19, 2013 at 11:55 AM, Bernerd Schaefer bern...@soundcloud.comwrote: Thanks for the response, Bill. Some followups below. I haven't found a great way to approach either of these in mesos without assuming that your framework has full control of the cluster. This is covered a bit in the Omega paper [1]: *While a Mesos framework can use “filters” to describe **the kinds of resources that it would like to be offered, it does **not have access to a view of the overall cluster state – just the **resources it has been offered. As a result, it cannot support **preemption or policies requiring access to the whole cluster **state: a framework simply does not have any knowledge **of resources that have been allocated to other schedulers.* I'm curious about your take as a framework author on the Omega paper's evaluation of Mesos. I would summarize their valuation somewhere between -- best-case -- Mesos is currently non-optimal for running a service scheduler alongside other schedulers, and -- worst-case -- Mesos is fundamentally unsuitable for service schedulers which do not own the entire cluster. The risk with this approach is that you wind up not playing nicely with other frameworks, possibly starving them of offers. Unfortunately this is the best way i've found to glean the shape of the cluster. Aurora cheats here by 'pinning' tasks to the same machines all the time, and (currently) not running anything else on those machines. Of course, this strategy falls apart when other frameworks are introduced. I believe mesos' reservations feature intends to address this. Given these comments -- am I to gather that Aurora runs on its own dedicated Mesos cluster? Regardless, it sounds like you've had to make Aurora itself a monolithic scheduler, which is discouraging. To my mind, the promise of Mesos is that I shouldn't have to build a scheduler that works for all-different kinds of tasks. I dream of a Mesos where my scheduler for stateless services lives happily alongside both my haproxy, memcached, elasticsearch schedulers, and my hadoop, spark, storm schedulers. Is it just not there yet? Bernerd Engineer @ SoundCloud
Re: Service Scheduling in Mesos
Thanks for the response, Bill. Some followups below. I haven't found a great way to approach either of these in mesos without assuming that your framework has full control of the cluster. This is covered a bit in the Omega paper [1]: *While a Mesos framework can use “filters” to describe **the kinds of resources that it would like to be offered, it does **not have access to a view of the overall cluster state – just the **resources it has been offered. As a result, it cannot support **preemption or policies requiring access to the whole cluster **state: a framework simply does not have any knowledge **of resources that have been allocated to other schedulers.* I'm curious about your take as a framework author on the Omega paper's evaluation of Mesos. I would summarize their valuation somewhere between -- best-case -- Mesos is currently non-optimal for running a service scheduler alongside other schedulers, and -- worst-case -- Mesos is fundamentally unsuitable for service schedulers which do not own the entire cluster. The risk with this approach is that you wind up not playing nicely with other frameworks, possibly starving them of offers. Unfortunately this is the best way i've found to glean the shape of the cluster. Aurora cheats here by 'pinning' tasks to the same machines all the time, and (currently) not running anything else on those machines. Of course, this strategy falls apart when other frameworks are introduced. I believe mesos' reservations feature intends to address this. Given these comments -- am I to gather that Aurora runs on its own dedicated Mesos cluster? Regardless, it sounds like you've had to make Aurora itself a monolithic scheduler, which is discouraging. To my mind, the promise of Mesos is that I shouldn't have to build a scheduler that works for all-different kinds of tasks. I dream of a Mesos where my scheduler for stateless services lives happily alongside both my haproxy, memcached, elasticsearch schedulers, and my hadoop, spark, storm schedulers. Is it just not there yet? Bernerd Engineer @ SoundCloud
Re: Service Scheduling in Mesos
Bernerd, You should really out Marathon https://github.com/mesosphere/marathon This fits closely for what you've described ;) Oh, I am! :) Perhaps I can shorten, clarify, and generalize my underlying questions. Assume I have a single framework (say, marathon) and its tasks occupy all but 5G of ram on each slave. 1. What happens if I ask the framework to start a task that wants 10G. Would this be a concern solely of the framework? Or would Mesos be able to intervene? For example, as far as I can tell marathon would be very unlikely to schedule this task, instead scheduling smaller tasks as resources become available. 2. What happens if I bring up a new framework which informs Mesos that it wants to schedule a 10G task. Will Mesos intervene (e.g., by killing some of the other framework's tasks)? Or will it only be scheduled after some number of tasks happen to finish? Bernerd Engineer @ SoundCloud
