GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/19227
[SPARK-20060][CORE] Support accessing secure Hadoop cluster in standalone
client mode
## What changes were proposed in this pull request?
This PR leverages the facility of SPARK-16742 to support accessing security
Hadoop cluster in standalone client mode. The key idea is brought from
SPARK-16742, add a simple logics to support `HadoopDelegationTokenManager` in
`StandaloneSchedulerBackend`. Besides this PR also address related
docs/configuration issue:
1. Rename "spark.yarn.principal", "spark.yarn.keytab" to
"spark.security.principal" and "spark.security.keytab", since principal/keytab
is not only used in yarn mode, so rename it to be more meaningful.
2. Move the related docs from `running-on-yarn` page to `security` page,
since it is not a yarn only feature.
3. Change `spark-submit --help` output to be more precise about
`--principal` and `--keytab`.
## How was this patch tested?
The change is verified in local security cluster.
@tgravescs @vanzin please help to review, thanks! I think the change is
quite straightforward, mainly on the doc/configuration things, update them to
be more accurate.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-20060
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19227.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19227
----
commit 2b3d2f24f94a1cee63fff9733b27f479673d7a90
Author: jerryshao <[email protected]>
Date: 2017-09-14T02:15:41Z
Support accessing secure Hadoop cluster in standalone client mode
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
