Would it be possible to have a mode where it tries to pull, but then does not fail solely due to the fail of a pull? In particular, we use tags to indicate which build should be deployed e.g. “foo-server:production” tag vs “foo-server:staging” tags.
On Sep 4, 2014, at 11:05 PM, Tim Chen <[email protected]> wrote: > Hi Maxime, > > It is a very valid concern and that's why I've added a patch that should go > out in 0.20.1 to not do a docker pull on every run anymore. > > Mesos will still try to docker pull when the image isn't available locally > (via docker inspect), but only once. > > The downside ofcourse is that you're not able to automatically get the latest > tagged image, but I think it's worth while price to may to gain the benefits > of not depending on registry, able to run local images and more. > > Tim > > > On Thu, Sep 4, 2014 at 10:50 PM, Maxime Brugidou <[email protected]> > wrote: > Hi, > > The current Docker integration in 0.20 does a "docker pull" from the registry > before running any task. This means that your entire Mesos cluster becomes > unusable if the registry goes down. > > The docs allow you to configure a custom .dockercfg for your tasks to point > to a private docker registry. > > However it is not easy to run an HA docker registry. The docker-registry > project recommend using S3 storage buy this is definitely not an option for > some people. > > I know that for regular artifacts, Mesos can use HDFS storage and you can run > your HDFS datanodes as Mesos tasks. > > So even if I attempt to have a docker registry storage in HDFS (which is not > supported by docker-registry at the moment), I am stuck on a chicken and egg > problem. I want to have as little services outside of Mesos as possible and > it is hard to maintain HA services (especially outside of Mesos). > > Is there anyone running Mesos with Docker in production without S3? I am > trying to make all the services outside of Mesos (the "infra" services that > are necessary to run Mesos like DNS, Haproxy, Chef server... etc) either HA > or not critical for the cluster to run. The docker registry is a new piece of > infra outside of Mesos that is critical... > > Best, > Maxime > >
