Hey, Thank you both for taking the time to respond. I wanted to be very expressive and give context to the points I was making, hopefully it wasn't too ranty. Ben, I've made some inline comments below.
I've had some chats with Ian Downes about doing something > like this for the MesosContainerizer too (but using a tarball/zip or path > that gets read-only bind-mounted into the container instead of a Docker > image). If we don't already have a JIRA issue for this specific feature, we > should add one so folks can follow along. This is very interesting, and can definitely see how that would be useful. We'd really like to > consolidate our own internal containerizers/isolation code with > libcontainer. That consolidation will likely be a long process, but it's > well worth it in the long run IMHO. I completely agree, though probably quite painful to change I can imagine Mesos making use of the same constructs as docker under the hood, and as you mentioned, completely sidestep the need to interact with the docker daemon in some cases. As you mention below, > this is a huge asset to Mesos and something I'd personally like to make > sure continues to work well, as would Till Toenshoff I'm sure. If you have > other suggestions for how to more generically support workflows we'd love > to hear them! Personally, from the discussions we've had here around what we plan to implement in the future and what we've been working on already the external containerizer API does indeed fit very well with zero adjustments, plus leveraging the ability to swap in our own logic where needed. Can you elaborate on what you mean when you say "unity configuration"? If > you're referring to command line flags, I'd love to hear your suggestions. > If you're referring to broader support for sharing components between > containerizers, I agree wholeheartedly. In fact, while the > containerizer/isolator part of the code base has seen quite a bit of > volatility in the last year I think it's due in for even more going > forward. Ultimately I'd like to see us reusing our existing isolators with > something like the Docker containerizer (and other containerizers) so that > folks can benefit from the isolation work that's being done irregardless of > how the container gets initially created! My mistake, that was meant to read "unify configuration". I think the containerizers could go a long way to sharing more concepts, and as you suggest, interact with each other irregardless of how the container is originally created (e.g container usage stats and limits, perhaps). I also feel the Docker Containerizer and the EC have diverged somewhat, especially around the ContainerInfo protobufs. I feel the EC could benefit from the more declarative options (i.e Volume's), though I do think it is necessary to support an arbitrary set of configuration options (key=value) that each specific containerizer could make use of, even if just until the option made its way top level. It would also be powerful to mix containerizers purely based on the ContainerInfo.TYPE field, automatically. If each containerizer were able to accept or reject responsibility based on a TaskInfo (kind of like how it works now, but more explicit), you could create more heterogeneous slaves. I am aware Mesos does some of these things, but I think there's some space to refine. We're very much looking forward to subsequent iterations of this area of Mesos, there's lots going on! :-) Tom. On 5 September 2014 19:49, Timothy Chen <[email protected]> wrote: > Hi Tom, > > As Ben mentioned it's definitely doable to introduce a default image > for docker. I was hesitant to use the existing default image flag in > the earlier point of the docker development as we didn't have > DockerInfo and was just reusing the ContainerInfo from the > CommandInfo, which makes it hard to distinguish between different > containerizers what it was meant for. > > Now with the new API it's definitely worth revisiting this and make > sure we have a good solution that is maintainable and also works well > with mulitple containerizers. > > I just created this, let's discuss more on the JIRA ticket: > https://issues.apache.org/jira/browse/MESOS-1768 > > Tim > > On Fri, Sep 5, 2014 at 9:50 AM, Benjamin Hindman <[email protected]> > wrote: > > I appreciate the thoughtful email Tom! There's a bunch there so I've just > > made some inline comments addressing your final questions. ;-) > > > > - Can the docker containerizer support more friendly defaults? If I only > >> want my mesos cluster to containerizer things with Docker, but don't > wish > >> to require every user specify an image for their tasks. > >> > > > > Requiring a user to specify an image for their task/executor was indeed > one > > of the biggest simplifying design decision we made for this first > iteration > > of Docker in Mesos. There was some discussion on JIRA about this too. > That > > all being said, I don't see any major blockers to introducing the idea > of a > > "default image" for a task/executor that doesn't specify any details in > > ContainerInfo. I've had some chats with Ian Downes about doing something > > like this for the MesosContainerizer too (but using a tarball/zip or path > > that gets read-only bind-mounted into the container instead of a Docker > > image). If we don't already have a JIRA issue for this specific feature, > we > > should add one so folks can follow along. > > > > - Since all of this Docker work has made it's way into the mesos core, a > >> lot of implicit decisions have been made about what options to support, > how > >> to expose them to users, and how the workflow looks for frameworks and > >> users. I think this is pretty limiting, and given Mesos is designed to > be > >> the fundamental building blocks for your datacenter, building for > specific > >> workflows concerns me. Is finding solutions to all of these workflows > >> really something the mesos-core team should focus on? > >> > > > > The explicit limiting was done because we wanted to abstract ourselves > away > > from the Docker CLI (as you mentioned earlier) and this is just the first > > iteration. Tim Chen is already working on the second iteration which > > includes things like bridge networking. Even so, we're really trying to > > abstract ourselves as much as possible since even Docker itself is being > > decomposed into libcontainer (and maybe more?) and we'd really like to > > consolidate our own internal containerizers/isolation code with > > libcontainer. That consolidation will likely be a long process, but it's > > well worth it in the long run IMHO. > > > > Regarding workflows, I still think the external containerizer is the best > > bet to capture an organization specific workflow. As you mention below, > > this is a huge asset to Mesos and something I'd personally like to make > > sure continues to work well, as would Till Toenshoff I'm sure. If you > have > > other suggestions for how to more generically support workflows we'd love > > to hear them! > > > > - I think the external containerizer is a value asset to Mesos even > without > >> Docker. There have been a few questions on the mailing list about using > >> other types of isolation (on windows for example). Could more work be > done > >> to unity configuration for each built-in containerizer? I feel like > lots of > >> different command line options might be a little confusing over time. > >> > > > > Can you elaborate on what you mean when you say "unity configuration"? If > > you're referring to command line flags, I'd love to hear your > suggestions. > > If you're referring to broader support for sharing components between > > containerizers, I agree wholeheartedly. In fact, while the > > containerizer/isolator part of the code base has seen quite a bit of > > volatility in the last year I think it's due in for even more going > > forward. Ultimately I'd like to see us reusing our existing isolators > with > > something like the Docker containerizer (and other containerizers) so > that > > folks can benefit from the isolation work that's being done irregardless > of > > how the container gets initially created! And as I mentioned above, it > > might make sense to directly consume libcontainer here too, possibly even > > contributing some of our isolators to it. > > > > Ben. >
