[
https://issues.apache.org/jira/browse/SPARK-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-6662.
----------------------------------
Resolution: Fixed
Fix Version/s: 1.4.0
Assignee: Cheolsoo Park
> Allow variable substitution in spark.yarn.historyServer.address
> ---------------------------------------------------------------
>
> Key: SPARK-6662
> URL: https://issues.apache.org/jira/browse/SPARK-6662
> Project: Spark
> Issue Type: Wish
> Components: YARN
> Affects Versions: 1.3.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Priority: Minor
> Labels: yarn
> Fix For: 1.4.0
>
>
> In Spark on YARN, explicit hostname and port number need to be set for
> "spark.yarn.historyServer.address" in SparkConf to make the HISTORY link. If
> the history server address is known and static, this is usually not a problem.
> But in cloud, that is usually not true. Particularly in EMR, the history
> server always runs on the same node as with RM. So I could simply set it to
> {{$\{yarn.resourcemanager.hostname\}:18080}} if variable substitution is
> allowed.
> In fact, Hadoop configuration already implements variable substitution, so if
> this property is read via YarnConf, this can be easily achievable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
