[
https://issues.apache.org/jira/browse/MESOS-816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklas Quarfot Nielsen updated MESOS-816:
-----------------------------------------
Description:
Being able to delegate isolation to shell scripts could make it easier to
leverage the machinery provided by the LXC tools, LibVirt, VirtualBox, Docker
and similar containerization systems.
Why go through command line tools for isolation? We have seen many requests for
isolation, covering a wide variety of scenarios:
- Setups requiring multiple versions of the same language (Ruby 1.8, Ruby 1.9).
- Setups requiring installation and configuration of RPM-packaged applications.
- Build-and-test setups, where sharing the environment of the host would impact
reproducibility.
- Integration of 3rd party, service-oriented applications.
- Launching applications with Docker.
- Launching multiple instances of a Mesos framework that, like Hadoop, has
significant system setup and dependencies.
To cover these and other use cases, it seems reasonable to allow Mesos to
delegate to external programs for isolation:
- It makes it easier to experiment with new containerization tools.
- It allows for site administrators to customize containerization, or even
implement new containerization mechanisms, without impacting their ability to
keep pace with Mesos development.
- Many external programs exist for containerization -- Docker, LXC tools,
LibVirt -- which handle a great deal of the book-keeping around finding and
efficiently cloning disk images and setting up the guest system (its hostname,
TTYs, /dev/*, /proc).
The scenarios listed above can be understood in terms of three use cases:
- The containerized system service scenario, wherein an application, installed
with RPM or a similar tool, is started and managed by the init system within a
container. Percona MySQL is an example of such an application.
- The containerized application scenario, wherein an application is installed
or unpacked and then configured and launched in a single command. For example,
running a custom Rails app with bundle install && bundle exec rails.
- The containerized framework/executor scenario, wherein the application is
Spark, Hadoop or another Mesos framework/executor pair.
One way to achieve this could be to introduce an External Isolator, which works
in parallel with the existing process/posix and cgroups isolators. The
responsibility of this isolator would be to act as a thin layer to external
isolators. Calls for task launching, stopping or any other resource change
would be serialized and passed to the external isolators by the Mesos External
Isolator.
We think an approach like this adds a lot of flexibility while still keeping a
good clean architecture and avoids using executors for isolation.
However, we are currently exploring how to solve this problem so feel free to
opt in with ideas, comments and suggestions.
was:
Delegating to shell scripts for isolation would make it easily to leverage the
machinery provided by the LXC tools, LibVirt, VirtualBox and similar systems.
Why go through command line tools for isolation? Such tools manage a great deal
of the book-keeping around finding and efficiently cloning disk images and
setting up the guest system (its hostname, TTYs, /dev/*, /proc) in a way that
is broadly useful.
A shell isolator would be called by the slave, with arguments allowing it to
ultimately start the executor under isolation.
One possible design is as follows:
* The slave calls the isolator with two arguments: a path to a Mesos boot strap
tool (a reimagining of mesos-launcher?) and a URL indicating how to connect to
the slave.
mesos-libvirt /usr/sbin/mesos-bootstrap localhost:5051
* The isolator sets up a container and runs mesos-bootstrap therein, copying it
(and attendant libraries?) as needed.
* The mesos-bootstrap tool connects to the slave and retrieves the ExecutorInfo
structure. It uses this information to launch the executor.
* The executor connects to the slave and receives TaskInfos.
Some questions arise with this design:
* How do we ensure libraries needed by mesos-bootstrap are present?
* What can we do to make it easier for the user to include libraries with the
executor? Should directories to copy or union mount in to the container be part
of the argument specification for the isolator?
> Allow delegation to shell scripts for isolation
> -----------------------------------------------
>
> Key: MESOS-816
> URL: https://issues.apache.org/jira/browse/MESOS-816
> Project: Mesos
> Issue Type: Improvement
> Components: isolation, slave
> Reporter: Jason Dusek
> Priority: Minor
> Attachments: mesos-shell-isolator.jpg
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Being able to delegate isolation to shell scripts could make it easier to
> leverage the machinery provided by the LXC tools, LibVirt, VirtualBox, Docker
> and similar containerization systems.
> Why go through command line tools for isolation? We have seen many requests
> for isolation, covering a wide variety of scenarios:
> - Setups requiring multiple versions of the same language (Ruby 1.8, Ruby
> 1.9).
> - Setups requiring installation and configuration of RPM-packaged
> applications.
> - Build-and-test setups, where sharing the environment of the host would
> impact reproducibility.
> - Integration of 3rd party, service-oriented applications.
> - Launching applications with Docker.
> - Launching multiple instances of a Mesos framework that, like Hadoop, has
> significant system setup and dependencies.
> To cover these and other use cases, it seems reasonable to allow Mesos to
> delegate to external programs for isolation:
> - It makes it easier to experiment with new containerization tools.
> - It allows for site administrators to customize containerization, or even
> implement new containerization mechanisms, without impacting their ability to
> keep pace with Mesos development.
> - Many external programs exist for containerization -- Docker, LXC tools,
> LibVirt -- which handle a great deal of the book-keeping around finding and
> efficiently cloning disk images and setting up the guest system (its
> hostname, TTYs, /dev/*, /proc).
> The scenarios listed above can be understood in terms of three use cases:
> - The containerized system service scenario, wherein an application,
> installed with RPM or a similar tool, is started and managed by the init
> system within a container. Percona MySQL is an example of such an application.
> - The containerized application scenario, wherein an application is installed
> or unpacked and then configured and launched in a single command. For
> example, running a custom Rails app with bundle install && bundle exec rails.
> - The containerized framework/executor scenario, wherein the application is
> Spark, Hadoop or another Mesos framework/executor pair.
> One way to achieve this could be to introduce an External Isolator, which
> works in parallel with the existing process/posix and cgroups isolators. The
> responsibility of this isolator would be to act as a thin layer to external
> isolators. Calls for task launching, stopping or any other resource change
> would be serialized and passed to the external isolators by the Mesos
> External Isolator.
> We think an approach like this adds a lot of flexibility while still keeping
> a good clean architecture and avoids using executors for isolation.
> However, we are currently exploring how to solve this problem so feel free to
> opt in with ideas, comments and suggestions.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
