dongjoon-hyun commented on PR #52650: URL: https://github.com/apache/spark/pull/52650#issuecomment-3498640554
Thank you for review, @littlexyw May I ask why you think in that way in the pointed code? `arguments.hostname` is `SPARK_EXECUTOR_POD_IP`, not a Spark Driver's K8s Service. Maybe, did you want to point out something else? https://github.com/apache/spark/blob/2ec7439f51f310cff2339c43648100d07318e6ee/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesExecutorBackend.scala#L129 > Spark Executor pod still connects to Spark Driver using Spark Driver's K8s Service while creating executor env in KubernetesExecutorBackend. By the way, since the service is no longer in use when we choose to use DriverPodIP, why not remove it under this case? This would ensure that all executor requests connect via IP address. <img alt="image" width="752" height="470" src="https://private-user-images.githubusercontent.com/157120263/510653122-82edac0c-01ec-4460-a313-b0eb25cef659.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjI0NDk1OTEsIm5iZiI6MTc2MjQ0OTI5MSwicGF0aCI6Ii8xNTcxMjAyNjMvNTEwNjUzMTIyLTgyZWRhYzBjLTAxZWMtNDQ2MC1hMzEzLWIwZWIyNWNlZjY1OS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUxMTA2JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MTEwNlQxNzE0NTFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01 NjkxNWNkMDViNGEzZDIxZGY2NzVhMTA0YWMxNDA5OGM3ZGVmMDRiM2Q3ZjFjZTE2MzAyMDk5N2JmOGYyYzYwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.UF-Ophf1m6Qzlx_Cbo50ALpzD77K8fnR2Ov2-Qe5ZeQ"> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
