haodemon commented on a change in pull request #33675:
URL: https://github.com/apache/spark/pull/33675#discussion_r686837436
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala
##########
@@ -92,7 +101,9 @@ private[spark] object SparkKubernetesClientFactory extends
Logging {
.withRequestTimeout(clientType.requestTimeout(sparkConf))
.withConnectionTimeout(clientType.connectionTimeout(sparkConf))
.withTrustCerts(sparkConf.get(KUBERNETES_TRUST_CERTIFICATES))
- .withOption(oauthTokenValue) {
+ .withOption(oauthTokenProviderInstance) {
+ (provider, configBuilder) =>
configBuilder.withOauthTokenProvider(provider)
+ }.withOption(oauthTokenValue) {
Review comment:
The token that was specified in `configBuilder.withOauthToken(token)`
will stay unused, and the class that implements `OAuthTokenProvider` will be
used instead. I confirmed this via tests on my cluster by simultaneously:
1 . Adding a debugging log into my class and providing it via
```
--conf spark.kubernetes.client.oauth.token.provider.class <myClass>
```
2. Specifying tokens via
```
--conf spark.kubernetes.authenticate.submission.oauthToken=<token>
--conf spark.kubernetes.authenticate.driver.oauthToken=<token>
--conf spark.kubernetes.authenticate.oauthToken=<token>
```
My log:
```
[2021-08-11 12:51:58,012] DEBUG (OAuthGoogleTokenProvider.java:62) -
Refreshing kubernetes oauth token via [/usr/lib/google-cloud-sdk/bin/gcloud,
config, config-helper, --format=json]
[2021-08-11 12:51:58,498] DEBUG (OAuthGoogleTokenProvider.java:73) - New
token expiry time is 3511s
```
From the log it could be concluded that `OAuthTokenProvider` has a higher
precedence over the token that was specified in
`configBuilder.withOauthToken(token) `. It could also be confirmed by looking
at the fabric's code in
https://github.com/fabric8io/kubernetes-client/blob/74cc63df9b6333d083ee24a6ff3455eaad0a6da8/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/RequestConfig.java#L136
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
