ecapoccia commented on issue #27034: [SPARK-30122][K8S] Support spark.kubernetes.authenticate.executor.serviceAccountName URL: https://github.com/apache/spark/pull/27034#issuecomment-581116487 Interestingly, I am reading this as I was about to submit an almost identical pull request. I should have checked in advance, too bad. In my implementation however the fallback for the executor service account name is not the driver's one but rather the default -- as far as I understand, this is the current functionality. The use case is quite simple, my worker nodes need to access AWS resources (primarily S3) and so far I had to use the node identity on the cluster nodes to set their roles. This violates the least privilege principle, as the role is now shared with every single pod than runs on the node. Hope this explain the reason why we worked out this solution @liyinan926 @dongjoon-hyun . I know in Spark 3 there is the plan to expose the entire pod template for the executor, which is a more reasonable solution than exposing the properties individually. However, we need to support the current applications running on Spark 2.4 (and in my case Scala 2.11 as I use Beam). Not sure what to do with my implementation thou, I will keep it for myself probably since this PR is already on its way. I think however that in order to be a like-for-like if the executor serviceAccountName is not set the default should be "default".
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
