GitHub user mgummelt opened a pull request:
https://github.com/apache/spark/pull/14275
[SPARK-16637] Unified containerizer
## What changes were proposed in this pull request?
New config var: spark.mesos.docker.containerizer={"mesos","docker"
(default)}
This adds support for running docker containers via the Mesos unified
containerizer: http://mesos.apache.org/documentation/latest/container-image/
The benefit is losing the dependency on `dockerd`, and all the costs which
it incurs.
I've also updated the supported Mesos version to 0.28.2 for support of the
required protobufs.
## How was this patch tested?
- manually testing jobs submitted with both "mesos" and "docker" settings
for the new config var.
- spark/mesos integration test suite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mesosphere/spark unified-containerizer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14275.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14275
----
commit 77bd01a89400fcf1a1481e134ea9900efb661b15
Author: Michael Gummelt <[email protected]>
Date: 2016-07-12T22:42:40Z
refactor
commit 5d37547f5c25dc42d2b0ecc299f1f8966b762eee
Author: Michael Gummelt <[email protected]>
Date: 2016-07-13T01:04:00Z
tests
commit f113ef7eb461b6230d195f92482a0580423719a1
Author: Michael Gummelt <[email protected]>
Date: 2016-07-13T19:00:43Z
whitespace fixes
commit c1f29689476490c2b59a895e69dba0f2236e1f3a
Author: Michael Gummelt <[email protected]>
Date: 2016-07-14T18:15:45Z
style changes
commit 8e0e4110ea8d1e2abe16340404e45e3266366dc6
Author: Michael Gummelt <[email protected]>
Date: 2016-07-14T19:08:16Z
rename to spark.mesos.driverEnv
commit 4aac0bafd820a66c5038d9dbec32f5366c3e623f
Author: Michael Gummelt <[email protected]>
Date: 2016-07-15T22:08:39Z
style
commit e0fe77363f6b050393389c532859ccc00529ce91
Author: Michael Gummelt <[email protected]>
Date: 2016-07-15T22:55:05Z
unified containerizer
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
