I see in the upcoming 0.23.0 there is support for oversubscription of
resources: the objective is to fill your cluster with background jobs
while still ensuring critical jobs have the resources they need. This is
great.
In order to implement this properly, it seems to me that the executor is
going to have to run the background and normal jobs differently (or run
them with different containerizer settings). For example, I expect that
the background jobs will need to be run at a lower CPU priority, so they
only use spare CPU time which is not required by the higher-priority
jobs. This presumably will be done by different cgroup settings,
depending on whether the offer contains revocable resources or not.
What's not clear to me is where this will be implemented, and whether
this needs further changes in either mesos or the frameworks running
under it. Does either the executor or the containerizer need to run
differently depending on whether the resources are revocable, and is it
up to the framework to tell it so?
Documentation in this area seems very hard to find.
For example, one of the frameworks (Aurora) has a detailed page on CPU
isolation:
http://aurora.apache.org/documentation/latest/resource-isolation/
But it's not made clear whether Aurora provides an external
containerizer of its own which implements this, or whether this page is
documenting standard Mesos functionality.
I can see that Mesos itself provides a "containerizer":
http://mesos.apache.org/documentation/latest/mesos-containerizer/
but that page doesn't even mention the two most important things you
might want to constrain, namely CPU usage and RAM usage! I have also
checked in git, and docs/mesos-containerizer.md doesn't mention whether
it could handle revocable and standard resources differently.
Mesos also provides a "docker containerizer":
http://mesos.apache.org/documentation/latest/docker-containerizer/
Apparently that shells out to the docker CLI. Again, it's not clear
whether it can tell the difference between revocable and standard
offers, nor indeed whether docker is able to run containers at different
priority levels.
So in summary: is 0.23.0 laying the foundations for all this, but with
more work required down the line? And does that work include changes to
frameworks as well as the core of mesos?
Regards,
Brian.