Hey Everyone,
Excited to see discussions of this. Something I started playing around with
just as the external containerizer was coming to life!
Diptanu – A few responses to your notes...
a. Mesos understanding docker metrics which should be straightforward because
docker writes all its metrics in the following fashin for cpu, blkio, memory
etc - /sys/fs/cgroup/cpu/docker/conntainerid
Are these paths not operating system dependent? I'm not too familiar with
cross-platform cgroups, so I am no doubt wrong here. These cgroup metrics are
also the ones Mesos currently uses (both for usage statistics and for
memory/cpu limits) so they can be pulled out much the same.
b. Easier way to map tasks to docker containers, probably the external
containerizer takes care of it to a large extent. It would be helpful if
there was a blog about it's API and internals by the core committers
explaining the design. Even a simple example in the mesos codebase using the
external containerizer would help.
That's an interesting one, a blog post would be awesome. The containerizers
right now currently use the ContainerId string provided to them from Mesos (I
believe this is just the TaskID but i'm not certain of that). This helps ensure
consistency of how containerizers are implemented, and makes them much simpler.
c. stdout and stderr of docker containers in the Mesos task stdout and stderr
logs. Does the external containerizer already takes care of it? I had to
write a service which runs on every slave for exposing the container logs to
an user.
The external containerizer itself doesn't help you with this. The logs from the
containerizer calls are dumped into the sandbox, however it's up to the
containerizer (e.g Deimos) to redirect the logs from the container it launches.
Deimos does take care of this already (as seen here
https://github.com/mesosphere/deimos/blob/master/deimos/containerizer/docker.py#L132).
e. Translate all task constraints to docker run flags. This is probably the
easiest and I know it's super easy to implement with the external
containerizer.
The current Docker containerizer implementations both do this, they support
CPU, Memory and Ports. Docker currently doesn't support changing these limits
on a running container, so you have to go behind docker and write to the
cgroup limits yourself. There's also no way to change the port mappings of a
container that I know of.
Hope that answers some of your questions!
Tom.
On 21 Jun 2014, at 00:20, Diptanu Choudhury dipta...@gmail.com wrote:
Great timing for this thread!
I have been working on this for the past few months and here is what I am
doing and would be nice if docker was supported straight way in Mesos. So
here goes the features that I would personally love to see in Mesos Core from
the perspective of an user which I had to implement on my own -
a. Mesos understanding docker metrics which should be straightforward because
docker writes all its metrics in the following fashin for cpu, blkio, memory
etc - /sys/fs/cgroup/cpu/docker/conntainerid
I am sending all these metrics right now as a framework message back to my
framework/scheduler but it would be cool if Mesos took care of them.
b. Easier way to map tasks to docker containers, probably the external
containerizer takes care of it to a large extent. It would be helpful if
there was a blog about it's API and internals by the core committers
explaining the design. Even a simple example in the mesos codebase using the
external containerizer would help.
c. stdout and stderr of docker containers in the Mesos task stdout and stderr
logs. Does the external containerizer already takes care of it? I had to
write a service which runs on every slave for exposing the container logs to
an user.
d. Mesos GC of tasks taking care of cleaning up docker containers which have
terminated. Right now the way I implemented this is that the service which
exposes the logs of a container also listens to docker events and when a
container exits, it knows that this has to be cleaned up and so removes it
after a fixed amount of time[configurable through a Rest API/config file]
e. Translate all task constraints to docker run flags. This is probably the
easiest and I know it's super easy to implement with the external
containerizer.
On Fri, Jun 20, 2014 at 3:40 PM, Tobias Knaup t...@knaup.me wrote:
Hi all,
We've got a lot of feedback from folks who use Mesos to run Dockers at scale
via Deimos, and the main wish was to make Docker a first class citizen in
Mesos, instead of a plugin that needs to be installed separately. Mesosphere
wants to contribute this and I already chatted with Ben H about what an
implementation could look like.
I'd love for folks on here that are working with Docker to chime in!
I created a JIRA here: https://issues.apache.org/jira/browse/MESOS-1524
Cheers,
Tobi
--
