Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19631#discussion_r149451751
--- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala ---
@@ -542,7 +496,55 @@ private[spark] class SecurityManager(
* Gets the secret key.
* @return the secret key as a String if authentication is enabled,
otherwise returns null
*/
- def getSecretKey(): String = secretKey
+ def getSecretKey(): String = {
+ if (isAuthenticationEnabled) {
+ Option(sparkConf.getenv(ENV_AUTH_SECRET))
+ .orElse(sparkConf.getOption(SPARK_AUTH_SECRET_CONF))
+ .getOrElse {
+ throw new IllegalArgumentException(
+ s"A secret key must be specified via the
$SPARK_AUTH_SECRET_CONF config")
+ }
+ } else {
+ null
+ }
+ }
+
+ /**
+ * Initialize the configuration object held by this class for
authentication.
+ *
+ * If authentication is disabled, do nothing.
+ *
+ * In YARN mode, generate a secret key and store it in the configuration
object, setting it up to
+ * also be propagated to executors using an env variable.
+ *
+ * In other modes, assert that the auth secret is set in the
configuration.
+ */
+ def initializeAuth(): Unit = {
+ if (!sparkConf.get(NETWORK_AUTH_ENABLED)) {
+ return
+ }
+
+ if (sparkConf.get(SparkLauncher.SPARK_MASTER, null) != "yarn") {
+ require(sparkConf.contains(SPARK_AUTH_SECRET_CONF),
+ s"A secret key must be specified via the $SPARK_AUTH_SECRET_CONF
config.")
+ return
+ }
+
+ // In YARN, force creation of a new secret if this is client mode.
This ensures each
--- End diff --
This behaves the same way as before for non-YARN. Standalone and Mesos have
always used hardcoded secrets in the config to authenticate executors to driver
and the driver to the master (in the case of standalone).
You can see the code I'm changing in this class, where for non-YARN it
would throw an error if the secret was not set. If changing that behavior is
desired for Mesos, then it should be done in a separate change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
