haodemon commented on a change in pull request #33675:
URL: https://github.com/apache/spark/pull/33675#discussion_r687304673
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala
##########
@@ -92,7 +101,9 @@ private[spark] object SparkKubernetesClientFactory extends
Logging {
.withRequestTimeout(clientType.requestTimeout(sparkConf))
.withConnectionTimeout(clientType.connectionTimeout(sparkConf))
.withTrustCerts(sparkConf.get(KUBERNETES_TRUST_CERTIFICATES))
- .withOption(oauthTokenValue) {
+ .withOption(oauthTokenProviderInstance) {
+ (provider, configBuilder) =>
configBuilder.withOauthTokenProvider(provider)
+ }.withOption(oauthTokenValue) {
Review comment:
> the current situation may cause a confusion to the Spark users.
Indeed. Thanks for making me realize this. The patch is missing a way for a
Spark user to specify how they would like the Spark to authenticate in
Kubernetes when running on a client or cluster mode. There is several options
present for oauthToken:
```
spark.kubernetes.authenticate.submission.oauthToken
spark.kubernetes.authenticate.driver.oauthToken
spark.kubernetes.authenticate.oauthToken
```
And I think we need to have the same for token provider, like
`spark.kubernetes.authenticate.*.oauthTokenProvider`.
This change would:
- Make the new options consistent with the rest of kubernetes options.
- Make existing options mutually exclusive. For every mode only one of the
`oauthToken`, `oauthTokenFile`, `oauthTokenProvider` would be allowed.
If we try this, we won't have to add anything about precedence into the docs
and there would be no need to ignore anything in the code.
@dongjoon-hyun, sorry for a lot of text.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
