[
https://issues.apache.org/jira/browse/YARN-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932529#comment-15932529
]
tuoyu commented on YARN-4970:
-----------------------------
Below is my properties in yarn-site.xml, and met the same problem. When the
job's status is "UNASSIGNED", through the driver host, i could access SparkUI
correctly, but when the job is scheduled by Yarn, neither $driver_host:$port or
Yarn's ApplicationMaster, all 500 error return.
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>h107710041.cluster.ds.weibo.com</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>h107710042.cluster.ds.weibo.com</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>h107710041.cluster.ds.weibo.com:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm1</name>
<value>h107710041.cluster.ds.weibo.com:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm2</name>
<value>h107710042.cluster.ds.weibo.com:8088</value>
</property>
I am not sure this problem could be around.
Any help, please, thanks.
> Difficult to trace "Connection Refused" in AM Proxying
> ------------------------------------------------------
>
> Key: YARN-4970
> URL: https://issues.apache.org/jira/browse/YARN-4970
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: webapp
> Affects Versions: 2.6.0
> Environment: Hadoop-2.6.0-CDH5.5.1, linux
> Reporter: Matthew Byng-Maddick
> Priority: Minor
> Labels: applicationmaster, proxy, webapp, yarn
>
> In generating an HA YARN config (effectively using my own tools for
> generation), I missed out the multiple specification of
> {{yarn.resourcemanager.webapp.address}}, which produced a "Connection
> Refused" similar to that in YARN-800 (because of a misunderstanding about
> what the proxy address was) from the AM webapp.
> It occurs to me, though, in tracing the code, that the behaviour of
> {{hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/RMHAUtils.java}}:
> {{getRMHAWebappAddresses(YarnConfiguration)}} should probably mirror the
> effect of the startup of the resourcemanager itself, such that if the
> id-suffixed key doesn't exist, you end up using the
> {{yarn.resourcemanager.address}} with the port number replaced by the one
> taken from {{yarn.resourcemanager.webapp.address}} (or its https equivalent).
> That is, if you had a config like:
> {code}
> <configuration>
> <property><name>yarn.resourcemanager.ha.enabled</name>
> <value>true</value></property>
> <property><name>yarn.resourcemanager.ha.rm-ids</name>
> <value>hosta,hostb</value></property>
> <property><name>yarn.resourcemanager.webapp.address</name>
> <value>0.0.0.0:8088</value></property>
> <property><name>yarn.resourcemanager.address.hosta</name>
> <value>hosta:8032</value></property>
> <property><name>yarn.resourcemanager.address.hostb</name>
> <value>hostb:8032</value></property>
> </configuration>
> {code}
> you would end up with ("{{hosta:8088}}", "{{hostb:8088}}") as the
> {{List<String>}} result of the function above, rather than the current empty
> list result.
> This would certainly give a better principle of least-surprise for operators
> of yarn (especially those like myself who ended up configuring it wrongly).
> Thoughts? Reasons why this isn't a good idea? (I'm afraid my Java's a bit
> read-only which is why I haven't suggested a patch)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
