apiwoni commented on PR #30684:
URL: https://github.com/apache/spark/pull/30684#issuecomment-2536641496
@holdenhk @dongjoon-hyun @hddong This PR created fixed and issue and created
another one because it breaks usage of `insecure-skip-tls-verify` via
KUBECONFIG file when using cluster context with auto configure.
Code below basically overwrites `insecure-skip-tls-verify` from KUBECONFIG
set by autoconfigure when KUBERNETES_TRUST_CERTIFICATES is missing. With this
bug it is impossible to drive hostname verification via KUBECONFIG when
multiple clusters are present with different requirements.
```
val KUBERNETES_TRUST_CERTIFICATES =
ConfigBuilder("spark.kubernetes.trust.certificates")
.createWithDefault(false)
val config = new ConfigBuilder(autoConfigure(kubeContext.orNull))
.withTrustCerts(sparkConf.get(KUBERNETES_TRUST_CERTIFICATES))
// kubernetes-client Config.autoConfigure
config.setTrustCerts(currentCluster.getInsecureSkipTlsVerify() != null &&
currentCluster.getInsecureSkipTlsVerify());
config.setDisableHostnameVerification(currentCluster.getInsecureSkipTlsVerify()
!= null && currentCluster.getInsecureSkipTlsVerify())
```
Correct implementation should do the following:
1. If KUBERNETES_TRUST_CERTIFICATES is not set AND Config from autoconfigure
isTrustCerts=true then no hostname verification
2. If KUBERNETES_TRUST_CERTIFICATES is not set AND Config from
autoconfigure isTrustCerts=false then do hostname verfication
3. If KUBERNETES_TRUST_CERTIFICATES is set use this as an override
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
