Hi Kevin! The ApplicationMaster doesn't really need any more configuration I think. Here's something to try out. Launch a very long mapreduce job:
# A sleep job with 1 mapper and 1 reducer. (All the mapper and reducer do is sleep for the duration specified in -mt and -rt) yarn jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3-tests.jar sleep -m 1 -r 1 -mt 9999999 -rt 9999999 Once the job starts running, follow the proxy URL. It should be served by the MapReduce ApplicationMaster. If you are able to attach debuggers to the RM, you can set breakpoints in https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java . I'm afraid the combinatorial space of all possible configuration is too huge to determine what is wrong with your cluster :( . HTH Ravi On Tue, Sep 12, 2017 at 10:27 PM, Kevin Buckley < kevin.buckley.ecs.vuw.ac...@gmail.com> wrote: > On 9 September 2017 at 05:17, Ravi Prakash <ravihad...@gmail.com> wrote: > > > I'm not sure my reply will be entirely helpful, but here goes. > > It sheds more light on things than I previously understood, Ravi, so cheers > > > The ResourceManager either proxies your request to the ApplicationMaster > (if > > the application is running), or (once the application is finished) > serves it > > itself if the job is in the "cache" (usually the last 10000 > applications) or > > redirects to the MapReduce JHS if its a MapReduce job. > > That suggests that I don't have the ApplicationMaster(s) setup correctly > (or possibly at all!) or that the caching setup is wrong because the > ResourceManager and JobHistoryServer clearly both has the Job Info > once the Jobs have ended, as I listed before > > > http://rmserver.ecs.vuw.ac.nz:8088/cluster/app/application_ > 1234567890123_4567/ > > > > Similarly, over on the Job History Server, we can get to a page, > > related to the job > > > > http://jhserver.ecs.vuw.ac.nz:19888/jobhistory/job/job_ > 1234567890123_4567/ > > however, trying to access those, though the "proxy channel" > > http://rmserver.ecs.vuw.ac.nz:8088/proxy/ > > URI doesn't take anyone anywhere. > > Thanks again for the insight: I think know where I need to look now, > Kevin >
