On 3 Mar 2016, at 09:17, Shady Xu <shad...@gmail.com<mailto:shad...@gmail.com>> wrote:
Hi all, I am running Spark in yarn-client mode, but every time I access the web ui, the browser redirect me to one of the worker nodes and shows nothing. The url looks like http://hadoop-node31.company.com:8088/proxy/application_1453797301246_120264 . that should be acting as a proxy for the web UI; the way YARN manages security and auth through the proxy; the spark Web UI runs a filter which redirects all queries that don't come from the proxy server to that proxy server It should really be picking up the address of that proxy from the hadoop cluster configuration option yarn.web-proxy.address , falling back to the value of yarn.resourcemanager.webapp.address if it can't find that value (it's actually a bit more complex than that, but you'd have to look at Hadoop's org.apache.hadoop.yarn.webapp.util.WebAppUtils for the full details I googled a lot and found some possible bugs of YARN which may cause this problem. But still want to know whether there is a way to show the web ui because it is very inconvenient to debug. It should be correctly linked through the yarn RM web UI, the one where you get a list of all running applications. If not: what about any other apps running in the cluster? Do they link up properly?
