Re: Docker support in Mesos core

2014-06-21 Thread Dick Davies
That's fantastic news, really good to see some integration happening
between chocolate and peanut butter
here. Deimos has been pretty difficult for us to deploy on our
platforms (largely down to the python implementation,
which has problems on the ancient python EL6 ships with).



On 20 June 2014 23:40, Tobias Knaup  wrote:
> Hi all,
>
> We've got a lot of feedback from folks who use Mesos to run Dockers at scale
> via Deimos, and the main wish was to make Docker a first class citizen in
> Mesos, instead of a plugin that needs to be installed separately. Mesosphere
> wants to contribute this and I already chatted with Ben H about what an
> implementation could look like.
>
> I'd love for folks on here that are working with Docker to chime in!
> I created a JIRA here: https://issues.apache.org/jira/browse/MESOS-1524
>
> Cheers,
>
> Tobi


Re: Docker support in Mesos core

2014-06-21 Thread Tom Arnfeld
Hey Everyone,

Excited to see discussions of this. Something I started playing around with 
just as the external containerizer was coming to life!

Diptanu – A few responses to your notes...

> a. Mesos understanding docker metrics which should be straightforward because 
> docker writes all its metrics in the following fashin for cpu, blkio, memory 
> etc - /sys/fs/cgroup/cpu/docker/

Are these paths not operating system dependent? I'm not too familiar with 
cross-platform cgroups, so I am no doubt wrong here. These cgroup metrics are 
also the ones Mesos currently uses (both for usage statistics and for 
memory/cpu limits) so they can be pulled out much the same.

> b. Easier way to map tasks to docker containers, probably the external 
> containerizer takes care of it to a large extent. It would be helpful if 
> there was a blog about it's API and internals by the core committers 
> explaining the design. Even a simple example in the mesos codebase using the 
> external containerizer would help.

That's an interesting one, a blog post would be awesome. The containerizers 
right now currently use the "ContainerId" string provided to them from Mesos (I 
believe this is just the TaskID but i'm not certain of that). This helps ensure 
consistency of how containerizers are implemented, and makes them much simpler.

> c. stdout and stderr of docker containers in the Mesos task stdout and stderr 
> logs. Does the external containerizer already takes care of it? I had to 
> write a service which runs on every slave for exposing the container logs to 
> an user.

The external containerizer itself doesn't help you with this. The logs from the 
containerizer calls are dumped into the sandbox, however it's up to the 
containerizer (e.g Deimos) to redirect the logs from the container it launches. 
Deimos does take care of this already (as seen here 
https://github.com/mesosphere/deimos/blob/master/deimos/containerizer/docker.py#L132).

> e. Translate all task constraints to docker run flags. This is probably the 
> easiest and I know it's super easy to implement with the external 
> containerizer.

The current Docker containerizer implementations both do this, they support 
CPU, Memory and Ports. Docker currently doesn't support changing these limits 
on a  running container, so you have to go behind docker and write to the 
cgroup limits yourself. There's also no way to change the port mappings of a 
container that I know of.

Hope that answers some of your questions!

Tom.

On 21 Jun 2014, at 00:20, Diptanu Choudhury  wrote:

> Great timing for this thread!
> 
> I have been working on this for the past few months and here is what I am 
> doing and would be nice if docker was supported straight way in Mesos. So 
> here goes the features that I would personally love to see in Mesos Core from 
> the perspective of an user which I had to implement on my own -
> 
> a. Mesos understanding docker metrics which should be straightforward because 
> docker writes all its metrics in the following fashin for cpu, blkio, memory 
> etc - /sys/fs/cgroup/cpu/docker/
> I am sending all these metrics right now as a framework message back to my 
> framework/scheduler but it would be cool if Mesos took care of them.
> 
> b. Easier way to map tasks to docker containers, probably the external 
> containerizer takes care of it to a large extent. It would be helpful if 
> there was a blog about it's API and internals by the core committers 
> explaining the design. Even a simple example in the mesos codebase using the 
> external containerizer would help.
> 
> c. stdout and stderr of docker containers in the Mesos task stdout and stderr 
> logs. Does the external containerizer already takes care of it? I had to 
> write a service which runs on every slave for exposing the container logs to 
> an user.
> 
> d. Mesos GC of tasks taking care of cleaning up docker containers which have 
> terminated. Right now the way I implemented this is that the service which 
> exposes the logs of a container also listens to docker events and when a 
> container exits, it knows that this has to be cleaned up and so removes it 
> after a fixed amount of time[configurable through a Rest API/config file]
> 
> e. Translate all task constraints to docker run flags. This is probably the 
> easiest and I know it's super easy to implement with the external 
> containerizer.
> 
> 
> On Fri, Jun 20, 2014 at 3:40 PM, Tobias Knaup  wrote:
> Hi all,
> 
> We've got a lot of feedback from folks who use Mesos to run Dockers at scale 
> via Deimos, and the main wish was to make Docker a first class citizen in 
> Mesos, instead of a plugin that needs to be installed separately. Mesosphere 
> wants to contribute this and I already chatted with Ben H about what an 
> implementation could look like.
> 
> I'd love for folks on here that are working with Docker to chime in!
> I created a JIRA here: https://issues.apache.org/jira/browse/MESOS-1524
> 
> Cheers,
> 
> Tobi
> 
> 
>