Jianshi Huang created SPARK-6277:
------------------------------------
Summary: Allow Hadoop configurations and env variables to be
referenced in spark-defaults.conf
Key: SPARK-6277
URL: https://issues.apache.org/jira/browse/SPARK-6277
Project: Spark
Issue Type: Improvement
Components: Spark Submit
Affects Versions: 1.2.1, 1.3.0
Reporter: Jianshi Huang
I need to set spark.local.dir to use user local home instead of /tmp, but
currently spark-defaults.conf can only allow constant values.
What I want to do is to write:
bq. spark.local.dir /home/${user.name}/spark/tmp
or
bq. spark.local.dir /home/${USER}/spark/tmp
Otherwise I would have to hack bin/spark-class and pass the option through
-Dspark.local.dir
Jianshi
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
