[
https://issues.apache.org/jira/browse/SPARK-30985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232601#comment-17232601
]
Dongjoon Hyun edited comment on SPARK-30985 at 11/16/20, 8:14 AM:
------------------------------------------------------------------
[~prashant]. You should not make a PR with the umbrella JIRA. The umbrella Jira
should be resolved when the all subtask JIRAs are resolved. However, if you
make a PR with the umbrella Jira ID, Spark merge script resolves it.
Since this seems to be created long time ago, the Jira management is up to you.
The above comment is just my suggestion.
was (Author: dongjoon):
[~prashant]. You should not make a PR with the umbrella JIRA. The umbrella Jira
is resolved when the all subtask JIRAs are resolved. If you make a PR with the
umbrella Jira ID, Spark merge script resolves it.
Since this seems to be created long time ago, the Jira management is up to you.
The above comment is just my suggestion.
> Propagate SPARK_CONF_DIR files to driver and exec pods.
> -------------------------------------------------------
>
> Key: SPARK-30985
> URL: https://issues.apache.org/jira/browse/SPARK-30985
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.1.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Major
> Fix For: 3.1.0
>
>
> SPARK_CONF_DIR hosts configuration files like,
> 1) spark-defaults.conf - containing all the spark properties.
> 2) log4j.properties - Logger configuration.
> 3) spark-env.sh - Environment variables to be setup at driver and executor.
> 4) core-site.xml - Hadoop related configuration.
> 5) fairscheduler.xml - Spark's fair scheduling policy at the job level.
> 6) metrics.properties - Spark metrics.
> 7) Any user specific - library or framework specific configuration file.
> Traditionally, SPARK_CONF_DIR has been the home to all user specific
> configuration files.
> So this feature, will let the user specific configuration files be mounted on
> the driver and executor pods' SPARK_CONF_DIR.
> Please review the attached design doc, for more details.
>
> [Google docs link|https://bit.ly/spark-30985]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
