[ https://issues.apache.org/jira/browse/SPARK-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves resolved SPARK-6662. ---------------------------------- Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Cheolsoo Park > Allow variable substitution in spark.yarn.historyServer.address > --------------------------------------------------------------- > > Key: SPARK-6662 > URL: https://issues.apache.org/jira/browse/SPARK-6662 > Project: Spark > Issue Type: Wish > Components: YARN > Affects Versions: 1.3.0 > Reporter: Cheolsoo Park > Assignee: Cheolsoo Park > Priority: Minor > Labels: yarn > Fix For: 1.4.0 > > > In Spark on YARN, explicit hostname and port number need to be set for > "spark.yarn.historyServer.address" in SparkConf to make the HISTORY link. If > the history server address is known and static, this is usually not a problem. > But in cloud, that is usually not true. Particularly in EMR, the history > server always runs on the same node as with RM. So I could simply set it to > {{$\{yarn.resourcemanager.hostname\}:18080}} if variable substitution is > allowed. > In fact, Hadoop configuration already implements variable substitution, so if > this property is read via YarnConf, this can be easily achievable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org