[
https://issues.apache.org/jira/browse/SPARK-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991367#comment-13991367
]
Shengzhe Yao commented on SPARK-1048:
-------------------------------------
[~sarutak] I am supposed to do the job but didn't figure out all use cases.
This could be as simple as add a fromFile method in SparkConf as [~velvia]
mentioned, but config file format should be addressed cleanly (like
Hadoop/HBase's xml config file or something else ?). I might to add a xml based
config file in next few weeks, but please feel free if you have ideas and want
to do it now. I am happy to see people like this idea: config file for Spark.
> Create spark-site.xml or spark-site.yaml for configuration
> ----------------------------------------------------------
>
> Key: SPARK-1048
> URL: https://issues.apache.org/jira/browse/SPARK-1048
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 0.8.1, 0.9.0, 1.0.0
> Reporter: Shengzhe Yao
> Priority: Critical
>
> Currently, spark doesn't provide the ability to pass configuration as a file,
> people have to explicitly specify them on the command line. This becomes a
> bigger issue when deploying spark on a cluster and people want to specify
> parameters other than default value.
> As of now, spark is being a top apache project and we need to pay more
> attention to the configuration. Most apache projects provide a xml
> configuration file (like hdfs, hadoop, hbase, etc.) and probably we want to
> do the same thing for spark. The advantages are obvious, it helps developers
> to specify their own spark configuration for the cluster and add/remove
> configuration parameters will be much easier via file than via system
> property.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
