[
https://issues.apache.org/jira/browse/SPARK-60345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Florian Müller updated SPARK-60345:
-----------------------------------
Description:
{{SSLOptions.parse}} falls back to the {{_SPARK_SSL_RPC_KEY_STORE_PASSWORD}},
{{_SPARK_SSL_RPC_KEY_PASSWORD}}, {{_SPARK_SSL_RPC_PRIVATE_KEY_PASSWORD}} and
{{_SPARK_SSL_RPC_TRUST_STORE_PASSWORD}} environment variables for every
namespace, not just {{spark.ssl.rpc.}} The lookup order is: the namespace's own
conf key, then the Hadoop credential provider, then the env variable, then the
{{spark.ssl.\*}} defaults. So whenever these variables are set in a process,
every enabled SSL namespace in that process resolves the RPC password ahead of
the password it would otherwise inherit from {{spark.ssl.\*}}. That includes
{{spark.ssl}} itself, when {{spark.ssl.keyStorePassword}} is not set.
Proposed fix: Only consult the environment variables when {{ns ==
"spark.ssl.rpc"}}. parse already special-cases that namespace for
{{enabledDefault}}. Within {{spark.ssl.rpc}}, keep the current lookup order.
was:
{{SSLOptions.parse}} falls back to the {{_SPARK_SSL_RPC_KEY_STORE_PASSWORD}},
{{_SPARK_SSL_RPC_KEY_PASSWORD}}, {{_SPARK_SSL_RPC_PRIVATE_KEY_PASSWORD}} and
{{_SPARK_SSL_RPC_TRUST_STORE_PASSWORD}} environment variables for every
namespace, not just {{spark.ssl.rpc.}} The lookup order is: the namespace's own
conf key, then the Hadoop credential provider, then the env variable, then the
{{spark.ssl.*}} defaults. So whenever these variables are set in a process,
every enabled SSL namespace in that process resolves the RPC password ahead of
the password it would otherwise inherit from {{spark.ssl.*}}. That includes
{{spark.ssl}} itself, when {{spark.ssl.keyStorePassword}} is not set.
Proposed fix: Only consult the environment variables when {{ns ==
"spark.ssl.rpc"}}. parse already special-cases that namespace for
{{enabledDefault}}. Within {{spark.ssl.rpc}}, keep the current lookup order.
> Scope the _SPARK_SSL_RPC_* environment fallback in SSLOptions to the RPC
> namespace
> ----------------------------------------------------------------------------------
>
> Key: SPARK-60345
> URL: https://issues.apache.org/jira/browse/SPARK-60345
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 4.3.0
> Reporter: Florian Müller
> Priority: Major
>
> {{SSLOptions.parse}} falls back to the {{_SPARK_SSL_RPC_KEY_STORE_PASSWORD}},
> {{_SPARK_SSL_RPC_KEY_PASSWORD}}, {{_SPARK_SSL_RPC_PRIVATE_KEY_PASSWORD}} and
> {{_SPARK_SSL_RPC_TRUST_STORE_PASSWORD}} environment variables for every
> namespace, not just {{spark.ssl.rpc.}} The lookup order is: the namespace's
> own conf key, then the Hadoop credential provider, then the env variable,
> then the {{spark.ssl.\*}} defaults. So whenever these variables are set in a
> process, every enabled SSL namespace in that process resolves the RPC
> password ahead of the password it would otherwise inherit from
> {{spark.ssl.\*}}. That includes {{spark.ssl}} itself, when
> {{spark.ssl.keyStorePassword}} is not set.
>
> Proposed fix: Only consult the environment variables when {{ns ==
> "spark.ssl.rpc"}}. parse already special-cases that namespace for
> {{enabledDefault}}. Within {{spark.ssl.rpc}}, keep the current lookup order.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
