Github user ArtRand commented on a diff in the pull request:
https://github.com/apache/spark/pull/19631#discussion_r149375835
--- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala ---
@@ -542,7 +496,55 @@ private[spark] class SecurityManager(
* Gets the secret key.
* @return the secret key as a String if authentication is enabled,
otherwise returns null
*/
- def getSecretKey(): String = secretKey
+ def getSecretKey(): String = {
+ if (isAuthenticationEnabled) {
+ Option(sparkConf.getenv(ENV_AUTH_SECRET))
+ .orElse(sparkConf.getOption(SPARK_AUTH_SECRET_CONF))
+ .getOrElse {
+ throw new IllegalArgumentException(
+ s"A secret key must be specified via the
$SPARK_AUTH_SECRET_CONF config")
+ }
+ } else {
+ null
+ }
+ }
+
+ /**
+ * Initialize the configuration object held by this class for
authentication.
+ *
+ * If authentication is disabled, do nothing.
+ *
+ * In YARN mode, generate a secret key and store it in the configuration
object, setting it up to
+ * also be propagated to executors using an env variable.
+ *
+ * In other modes, assert that the auth secret is set in the
configuration.
+ */
+ def initializeAuth(): Unit = {
+ if (!sparkConf.get(NETWORK_AUTH_ENABLED)) {
+ return
+ }
+
+ if (sparkConf.get(SparkLauncher.SPARK_MASTER, null) != "yarn") {
+ require(sparkConf.contains(SPARK_AUTH_SECRET_CONF),
+ s"A secret key must be specified via the $SPARK_AUTH_SECRET_CONF
config.")
+ return
+ }
+
+ // In YARN, force creation of a new secret if this is client mode.
This ensures each
--- End diff --
Is there a reason this _has_ to be unique to YARN? Will this solve the
problem (in Mesos currently) where when the Executors
[bootstrap](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala#L193)
they do so without security (unless you
"[bake](https://github.com/apache/spark/blob/e1960c3d6f380b0dfbba6ee5d8ac6da4bc29a698/core/src/main/scala/org/apache/spark/SparkConf.scala#L482)"
the secret and secret config into the container image)? Looks like propagating
the envvar is only handled in the YARN case?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
