Robert , Thanks for the response. If I do not provide any explicit configuration for the proxy server, do I still need to start it using the 'yarn start proxy server'? I am currently not doing it.
Also, I am able to access the html page for proxy using the <http://localhost:8088/proxy/{appid}/mapreduce/jobs> URL. (Note this url does not have the '/ws/v1/ part in it. I get the html response when I query for this URL in runtime. So I assume the proxy server must be starting fine since I am able to access this URL. I will try logging more details tomorrow from my office machine and will let you know the result. Regards, Prajakta On Fri, Jul 6, 2012 at 10:22 PM, Robert Evans <ev...@yahoo-inc.com> wrote: > Sorry I did not respond sooner. The default behavior is to have the proxy > server run as part of the RM. I am not really sure why it is not doing > this in your case. If you set the config yourself to be a URI that is > different from that of the RM then you need to launch a standalone proxy > server. You can do this by running > > yarn start proxy server > > Without sitting down with you it is going to be somewhat difficult to > debug why this is happening. However, in retrospect it would be nice to > add in some extra logging to help indicate why the proxy server is not > functioning as desired. If you could file a JIRA to add in the logging I > would be happy to provide a patch to you and we can try and debug the > issue further. Please file it under the MAPREDUCE JIRA project. > > --Bobby > > On 7/6/12 3:29 AM, "Prajakta Kalmegh" <pkalm...@gmail.com> wrote: > > >Re-posting as I haven't got a solution yet. Sorry for spamming. I won't be > >able to proceed in my code until I get a JSON response using AppMaster > >REST > >URL. :( > > > >Thanks, > >Prajakta > > > > > >On Wed, Jul 4, 2012 at 5:55 PM, Prajakta Kalmegh <pkalm...@gmail.com> > >wrote: > > > >> Hi Robert/Harsh > >> > >> Thanks for your reply. > >> > >> My RM is starting just fine. The problem is with the use of > >>http://<proxy httpddress:port>/proxy/{appid}/ws/v1/mapreduce > >> to get the JSON response. > >> > >> As I said before, I had not configured the yarn.web-proxy.address > >>property in yarn-site.xml. I assumed it will use the RM's > >>yarn.resourcemanager.webapp.address property value as default. However, > >>it gives me a '404-Page not found error'. Today I tried specifying a > >>value explicitly for the yarn.web-proxy.address property. > >> > >> On running the wordcount example, it even gives a url > >><http://localhost:8090>/proxy/{appid}/> to track the App Mast info. > >>However, I am still not able to get a json response. > >> > >> Also, I tried to get the data from historyserver instead of runtime > >>using the instructions given on page > >>< > http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yar > >>n-site/HistoryServerRest.html> > >> > >> HistoryServer REST response does not give me jobids corresponding to an > >>application. It just lists all the jobs run until now. By the way, the > >>documentation does say > >> > >> ---------- > >> > >> "Both of the following URI's give you the history server information, > >>from an application id identified by the appid value. > >> * http://<history server http address:port>/ws/v1/history > >> * http://<history server http address:port>/ws/v1/history/info" > >> --------- > >> > >> But there is no provision to specify the application id with these REST > >>URLs. > >> > >> Any idea how I can get the Application Master REST working and also > >>linking jobids to application id using the HistoryServerREST API? > >> > >> Any help is appreciated. Thanks in advance. > >> Regards, > >> Prajakta > >> > >> > >> > >> > >> On Fri, Jun 29, 2012 at 8:55 PM, Robert Evans <ev...@yahoo-inc.com> > >>wrote: > >> > >>> Please don't file that JIRA. The proxy server is intended to front the > >>> web server for all calls to the AM. This is so you only have to go to > >>>a > >>> single location to get to any AM's web service. The proxy server is a > >>> very simple proxy and just forwards the extra part of the path on to > >>>the > >>> AM. > >>> > >>> If you are having issues with this please include the version you are > >>> having problems with. Also please look at the logs for the RM on > >>>startup > >>> to see if there is anything there indicating why it is not starting up. > >>> > >>> --Bobby Evans > >>> > >>> On 6/28/12 9:46 AM, "Harsh J" <ha...@cloudera.com> wrote: > >>> > >>> >As far as I can tell, the MR WebApp, as the name itself indicates on > >>> >its doc page, starts only at the MR AM (which may be running at any > >>> >NM), and it starts as an ephemeral port logged at in the AM logs > >>> >usually as: > >>> > > >>> >INFO Web app /mapreduce started at [PORT] > >>> > > >>> >That it starts its own server with an ephemeral access point makes > >>> >sense, since each job uses its own AM and having a common location may > >>> >not work with the form of REST API documented at your link. Can you > >>> >please file a JIRA to fix the doc and remove the proxy server refs, > >>> >which are misleading? > >>> > > >>> >Do correct me if I'm wrong. > >>> > > >>> >On Thu, Jun 28, 2012 at 6:13 PM, Prajakta Kalmegh <pkalm...@gmail.com > > > >>> >wrote: > >>> >> Hi > >>> >> > >>> >> I am trying to get the ApplicationMaster info using the > >>><http://<proxy > >>> >>http > >>> >> address:port>/proxy/{appid}/ws/v1/mapreduce/info> link as described > >>>on > >>> >>the < > >>> >> > >>> >> > >>> > >>> > http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yar > >>>n > >>> >>-site/MapredAppMasterRest.html> > >>> >> page. > >>> >> > >>> >> I am able to access and retrieve JSON response for other modules > >>> >> (ResourceManager, NodeManager and HistoryServer). However, I am > >>>getting > >>> >> 'Page not found' when I try to use my ResourceManager Http address > >>>to > >>> >> access the ApplicationMaster info. I am using < > >>> >> http://localhost:8088/proxy/{appid}/ws/v1/mapreduce/info> to > >>>retrieve > >>> >>JSON > >>> >> response. > >>> >> > >>> >> The instructions say "The application master should be accessed via > >>>the > >>> >> proxy. This proxy is configurable to run either on the resource > >>>manager > >>> >>or > >>> >> on a separate host." > >>> >> > >>> >> My yarn-default.xml contains: > >>> >> <property> > >>> >> <description>The address for the web proxy as HOST:PORT, if this > >>>is > >>> >>not > >>> >> given then the proxy will run as part of the RM</description> > >>> >> <name>yarn.web-proxy.address</name> > >>> >> <value/> > >>> >> </property> > >>> >> > >>> >> and I did not set a value explicitly in yarn-site.xml. Any idea > >>>how I > >>> >>can > >>> >> get this working? Thanks in advance. > >>> >> > >>> >> Regards, > >>> >> Prajakta > >>> > > >>> > > >>> > > >>> >-- > >>> >Harsh J > >>> > >>> > >> > >
