We ended up implementing a solution where we did the pre/post steps as separate mesos tasks and adding logic to our scheduler to ensure they were run on the same machine. If anybody knows of a standard / openly available DockerExecutor like what is described below, my team would be greatly interested.
On Fri, Aug 7, 2015 at 4:01 AM, Kapil Malik <[email protected]> wrote: > Hi, > > > > We have a similar usecase while running multi-user workloads on mesos. > Users provide docker images encapsulating application logic, which we (we = > say some “Central API”) schedule on Chronos / Marathon. However, we need to > run some standard pre / post steps for every docker submitted by users. We > have following options – > > > > 1. Ask every user to embed their logic inside a pre-defined docker > template which will perform pre/post steps. > > è This is error prone, makes us dependent on whether the users followed > template, and not very popular with users either. > > > > 2. Extend every user docker (FROM <>) and find a way to add > pre-post steps in our docker. Refer this docker when scheduling on chronos > / marathon. > > è Building new dockers does not scale as users and applications grow > > > > 3. Write a custom executor which will perform the pre-post steps > and manage the user docker lifetime. > > è Deals with user docker lifetime and is obviously complex. > > > > Is there a standard / openly available DockerExecutor which manages the > docker lifetime and which I can extend to build my custom executor? This > way I will be concerned only with my custom logic (pre/post steps) and > still get benefits of a standard way to manage docker containers. > > > > Btw, thanks for the meaningful discussion below, it is very helpful. > > > > Thanks and regards, > > > > Kapil Malik | [email protected] | 33430 / 8800836581 > > > > *From:* James DeFelice [mailto:[email protected]] > *Sent:* 09 April 2015 18:12 > *To:* [email protected] > *Subject:* Re: Custom python executor with Docker > > > > If you can run the pre/post steps in a container then I'd recommend > building a Docker image that includes your pre/post step scripting + your > algorithm and launching it using the built-in mesos Docker containerizer. > It's much simpler than managing the lifetime of the Docker container > yourself. > > > > On Thu, Apr 9, 2015 at 8:29 AM, Tom Fordon <[email protected]> wrote: > > Thanks for all the responses, I really appreciate the help. Let me try to > state my problem more clearly > > > > Our project is performing file-based data processing. I would like to > keep the actual algorithm as contained as possible since we are in an R&D > setting and will be getting untested code. We have some pre/post steps > that need to be run on the same box as the actual algorithm: > downloading/uploading files and database calls. > > > > We can run the pre/post steps and algorithm within the same container. > The algorithm will be a little less contained, but it will work. > > > > Docker letting you specify a cgroup parent is really exciting. If I > invoke a docker container with the executor as the cgroup-parent are there > any other steps I need to perform? Would I need to do anything special to > make mesos aware of the resource usage, or is that handled since the docker > process would be in the executors cgroup? > > > > Thanks again, > > Tom > > > > On Tue, Apr 7, 2015 at 8:10 PM, Timothy Chen <[email protected]> wrote: > > Hi Tom(s), > > Tom Arnfeld is right, if you want to launch your own docker container > in your custom executor you will have to handle all the issues > yourself and not able to use the Docker containerizer at all. > > Alternatively, you can actually launch your custom executor in a > Docker container by Mesos, by specifying the ContainerInfo in the > ExecutorInfo. > What this means is that your custom executor is already running in a > docker container, and you can do your custom logic afterwards. This > does means you can simply just launch multiple containers in the > executor anymore. > > If there is something you want to do and doesnt' fit these let us know > what you're trying to achieve and we can see what we can do. > > Tim > > On Tue, Apr 7, 2015 at 4:15 PM, Tom Arnfeld <[email protected]> wrote: > > > It's not possible to invoke the docker containerizer from outside of > Mesos, > > as far as I know. > > > > If you persue this route, you can run into issues with orphaned > containers > > as your executor may die for some unknown reason, and the container is > still > > running. Recovering from this can be tricky business, so it's better if > you > > can adapt your framework design to fit within the Mesos Task/Executor > > pattern. > > > > -- > > > > Tom Arnfeld > > Developer // DueDil > > > > (+44) 7525940046 > > 25 Christopher Street, London, EC2A 2BS > > > > > > On Mon, Apr 6, 2015 at 7:00 PM, Vinod Kone <[email protected]> wrote: > >> > >> Tim, do you want answer this? > >> > >> On Wed, Apr 1, 2015 at 7:27 AM, Tom Fordon <[email protected]> > wrote: > >>> > >>> Hi. I'm trying to understand using docker within a custom executor. > For > >>> each of my tasks, I would like to perform some steps on the node before > >>> launching a docker container. I was planning on writing a custom python > >>> executor for this, but I wasn't sure how to launch docker from within > this > >>> executor. > >>> > >>> Can I just call docker in a subprocess using the ContainerInfo from the > >>> Task? If I do this, how does the Containerizer fit in? > >>> > >>> Thank you, > >>> Tom Fordon > >> > >> > > > > > > > > > > -- > > James DeFelice > 585.241.9488 (voice) > 650.649.6071 (fax) >
