Erik Erlandson created SPARK-24435:
--------------------------------------
Summary: Support user-supplied YAML that can be merged with k8s
pod descriptions
Key: SPARK-24435
URL: https://issues.apache.org/jira/browse/SPARK-24435
Project: Spark
Issue Type: New Feature
Components: Kubernetes
Affects Versions: 2.3.0
Reporter: Erik Erlandson
Fix For: 2.4.0
Kubernetes supports a large variety of configurations to Pods. Currently only
some of these are configurable from Spark, and they all operate by being
plumbed from --conf arguments through to pod creation in the code.
To avoid the anti-pattern of trying to expose an unbounded Pod feature set
through Spark configuration keywords, the community is interested in working
out a sane way of allowing users to supply "arbitrary" Pod YAML which can be
merged with the pod configurations created by the kube backend.
Multiple solutions have been considerd, including Pod Pre-sets and loading Pod
template objects. A requirement is that the policy for how user-supplied YAML
interacts with the configurations created by the kube back-end must be easy to
reason about, and also that whatever kubernetes features the solution uses are
supported on the kubernetes roadmap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
