[
https://issues.apache.org/jira/browse/SPARK-32223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238617#comment-17238617
]
Prashant Sharma commented on SPARK-32223:
-----------------------------------------
Hi [~dongjoon],
Do you think this is useful ? Any other thoughts on this?
Thanks !
> Support adding a user provided config map.
> ------------------------------------------
>
> Key: SPARK-32223
> URL: https://issues.apache.org/jira/browse/SPARK-32223
> Project: Spark
> Issue Type: Sub-task
> Components: Kubernetes
> Affects Versions: 3.1.0
> Reporter: Prashant Sharma
> Priority: Major
>
> One of the challenge with this is, spark.properties is not user provided and
> is calculated based on certain factors. So a user provided config map, cannot
> be used as is to mount as SPARK_CONF_DIR, so it will have to be somehow
> augmented with the correct spark.properties.
> Q, Do we support update to config map properties for an already running job?
> Ans: No, since the spark.properties is calculated at the time of job
> submission, it cannot be updated on the fly and it is not supported by Spark
> at the moment for all the configuration values.
> Q. What are the usecases, where supplying SPARK_CONF_DIR via a config map
> helps?
> One of the use case, I can think of is programmatically submitting a spark on
> k8s job - e.g. spark as a service on cloud deployment may find this feature
> useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
