[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498307#comment-16498307 ]
Yinan Li edited comment on SPARK-24434 at 6/1/18 5:39 PM: ---------------------------------------------------------- The pod template is basically a pod specification and can contain every possible pieces of information about a pod. It should look similar to what the core workload types (deployments and statefulsets for example) use, which contains a {{[PodSpec|https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/api/core/v1/types.go#L2636]}}. The problem is unique for the Kubernetes mode as there are many things to customize for a pod. Currently we basically just introduce a new Spark config property for each new customization aspect of a pod. Given the number of things to customize, this will soon become hard to maintain if we keep introducing new config properties. was (Author: liyinan926): The pod template is basically a pod specification and can contain every possible pieces of information about a pod. It should look similar to what the core workload types (deployments and statefulsets for example) use, which contains a {{[PodSpec|https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/api/core/v1/types.go#L2636]}}. The problem is unique for the Kubernetes mode as there are many things to customize for a pod. Currently we basically just introduce a new Spark config property for each new customization aspect of a pod. Given the number of things to customize, this will soon become hard to maintain if we keep introducing new config properties. > Support user-specified driver and executor pod templates > -------------------------------------------------------- > > Key: SPARK-24434 > URL: https://issues.apache.org/jira/browse/SPARK-24434 > Project: Spark > Issue Type: New Feature > Components: Kubernetes > Affects Versions: 2.4.0 > Reporter: Yinan Li > Priority: Major > > With more requests for customizing the driver and executor pods coming, the > current approach of adding new Spark configuration options has some serious > drawbacks: 1) it means more Kubernetes specific configuration options to > maintain, and 2) it widens the gap between the declarative model used by > Kubernetes and the configuration model used by Spark. We should start > designing a solution that allows users to specify pod templates as central > places for all customization needs for the driver and executor pods. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org