littlexyw commented on PR #52927: URL: https://github.com/apache/spark/pull/52927#issuecomment-3500899353
> Thank you, @littlexyw . However, as I mentioned in the JIRA issue, this is invalid in two ways. > > 1. Apache Spark provides the same capability already via "spark.kubernetes.driver.pod.excludedFeatureSteps=org.apache.spark.deploy.k8s.features.DriverServiceFeatureStep". > 2. A user still need K8s Driver Service for Spark UI like this while using `useDriverPodIP=true`. > > ``` > $ kubectl port-forward svc/spark-connect-server-0-driver-svc 4040 > Forwarding from 127.0.0.1:4040 -> 4040 > Forwarding from [::1]:4040 -> 4040 > ``` > > They should be handled orthogonally via `useDriverPodIP` and `excludedFeatureSteps`. The primary purpose of the `useDriverPodIP` feature is to prevent systemic impacts on Spark from Kubernetes CoreDNS failures. Enabling this feature allows the driver to communicate directly using the Pod's IP address without starting a corresponding DNS service. Furthermore, users can directly access the Spark UI via the driver Pod's IP address, resulting in a simpler and more stable deployment solution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
