GitHub user susanxhuynh opened a pull request:
https://github.com/apache/spark/pull/19437
[SPARK-22131][MESOS] Mesos driver secrets
## Background
In #18837 , @ArtRand added Mesos secrets support to the dispatcher. **This
PR is to add the same secrets support to the drivers.** This means if the
secret configs are set, the driver will launch executors that have access to
either env or file-based secrets.
One use case for this is to support TLS in the driver <=> executor
communication.
## What changes were proposed in this pull request?
Most of the changes are a refactor of the dispatcher secrets support
(#18837) - moving it to a common place that can be used by both the dispatcher
and drivers. The same goes for the unit tests.
## How was this patch tested?
There are four config combinations: [env or file-based] x [value or
reference secret]. For each combination:
- Added a unit test.
- Tested in DC/OS.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mesosphere/spark sh-mesos-driver-secret
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19437.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19437
----
commit b289bcc95f0b67cda94ddf416fc9a15e5d1855b4
Author: Susan X. Huynh <[email protected]>
Date: 2017-10-04T11:30:31Z
[SPARK-22131] Mesos driver secrets. The driver launches executors that have
access to env or file-based secrets.
commit 6f062c00f6382d266619b4a56a753ec27d1db10b
Author: Susan X. Huynh <[email protected]>
Date: 2017-10-05T12:07:20Z
[SPARK-22131] Updated docs
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
