[ https://issues.apache.org/jira/browse/SPARK-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-6662: ----------------------------------- Assignee: Apache Spark > Allow variable substitution in spark.yarn.historyServer.address > --------------------------------------------------------------- > > Key: SPARK-6662 > URL: https://issues.apache.org/jira/browse/SPARK-6662 > Project: Spark > Issue Type: Wish > Components: YARN > Affects Versions: 1.3.0 > Reporter: Cheolsoo Park > Assignee: Apache Spark > Priority: Minor > Labels: yarn > > In Spark on YARN, explicit hostname and port number need to be set for > "spark.yarn.historyServer.address" in SparkConf to make the HISTORY link. If > the history server address is known and static, this is usually not a problem. > But in cloud, that is usually not true. Particularly in EMR, the history > server always runs on the same node as with RM. So I could simply set it to > {{$\{yarn.resourcemanager.hostname\}:18080}} if variable substitution is > allowed. > In fact, Hadoop configuration already implements variable substitution, so if > this property is read via YarnConf, this can be easily achievable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org