[
https://issues.apache.org/jira/browse/SPARK-24174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-24174:
---------------------------------
Labels: bulk-closed features usability (was: features usability)
> Expose Hadoop config as part of /environment API
> ------------------------------------------------
>
> Key: SPARK-24174
> URL: https://issues.apache.org/jira/browse/SPARK-24174
> Project: Spark
> Issue Type: Wish
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: Nikolay Sokolov
> Priority: Minor
> Labels: bulk-closed, features, usability
>
> Currently, UI or /environment API call of HistoryServer or WebUI exposes only
> system properties and SparkConf. However, in some cases when Spark is used in
> conjunction with Hadoop, it is useful to know Hadoop configuration
> properties. For example, HDFS or GS buffer sizes, hive metastore settings,
> and so on.
> So it would be good to have hadoop properties being exposed in /environment
> API, for example:
> {code:none}
> GET .../application_1525395994996_5/environment
> {
> "runtime": {"javaVersion": "1.8.0_131 (Oracle Corporation)", ...}
> "sparkProperties": ["java.io.tmpdir","/tmp", ...],
> "systemProperties": [["spark.yarn.jars", "local:/usr/lib/spark/jars/*"],
> ...],
> "classpathEntries": [["/usr/lib/hadoop/hadoop-annotations.jar","System
> Classpath"], ...],
> "hadoopProperties": [["dfs.stream-buffer-size", 4096], ...],
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
