Hi, Tushar. Thanks. Yes. It worked.
I have one more question. When I access to hadoop web UI and there is menu "ApplicationMaster" in Tracking UI menu. When I click this, it gave me error. Sorry, got error 404 Please consult RFC 2616 for meanings of the error code. Error Details org.apache.hadoop.yarn.webapp.WebAppException: /: controller for default not found at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232) at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140) at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795) at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) The URL is "http://<IP address>:8088/proxy/application_1464766676472_0002/. With this menu, what we can see? Thanks, Junguk 2016-06-01 1:24 GMT-06:00 Tushar Gosavi <[email protected]>: > Hi Junguk, > > Please check your yarn configuration settings. I normally set > minimum-allocation-mb to 256MB for single node install, you can add > following lines to your yarn-site.xml and restart the yarn. > > <property> > <name>yarn.scheduler.minimum-allocation-mb</name> > <value>256</value> > </property> > > I am able to run YahooFinanceDemo app with above setting on my laptop > with 5GB memory usage. > > Regards, > -Tushar. > > On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh > <[email protected]> wrote: >> Thanks for details. >> >> Typically the "Waiting for pending status" is followed by "Deploy request" >> as highlighted below. Are you getting the highlighted line in your log? If >> not, the operators is not getting deployed correctly. Please let us know. >> >> >> 2016-05-23 14:46:51,649 DEBUG >> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512 >> 2016-05-23 14:46:51,655 INFO >> com.datatorrent.bufferserver.storage.DiskStorage: using >> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp >> as the basepath for spooling. >> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server: >> Server started listening at /0:0:0:0:0:0:0:0:42358 >> 2016-05-23 14:46:51,665 DEBUG >> com.datatorrent.stram.engine.StreamingContainer: Buffer server started: >> /0:0:0:0:0:0:0:0:42358 >> 2016-05-23 14:46:51,700 DEBUG >> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop >> (interval is 1000 ms) >> 2016-05-23 14:46:52,750 INFO >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >> request. >> 2016-05-23 14:46:53,255 INFO >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >> request. >> 2016-05-23 14:46:56,287 INFO >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >> request. >> 2016-05-23 14:46:56,791 INFO >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >> request. >> 2016-05-23 14:46:57,295 INFO >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >> request. >> 2016-05-23 14:46:57,809 INFO >> com.datatorrent.stram.engine.StreamingContainer: Deploy request: >> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff, >> 0, >> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]] >> 2016-05-23 14:46:57,811 DEBUG >> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to >> checkpoint ffffffffffffffff stateless=false. >> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT, >> operator class=class com.datatorrent.InputFileReader >> 2016-05-23 14:46:57,889 DEBUG >> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as >> deployed. >> >> >> Regards, >> Sandeep >> >> Regards, >> Sandeep >> >> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <[email protected]> wrote: >>> >>> Hi, >>> >>> 1. I used only one machine and use docker image to run it. >>> When I access to ip:8088, it shows I use 8 GB memory. >>> (Is there a configuration file I can check?) >>> >>> 2. Hadoop version : 2.7.1 >>> I tested both Apex 3.2.2 and Apex 3.3. >>> >>> Thanks, >>> Junguk >>> >>> >>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <[email protected]>: >>> > Hi Junguk, >>> > >>> > Could you please give us some details: >>> > >>> > Hadoop cluster configuration: no of machines and their configuration >>> > (memory >>> > in particular) >>> > Version of Hadoop and Apex >>> > >>> > Regards, >>> > Sandeep >>> > >>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <[email protected]> wrote: >>> >> >>> >> Hi, >>> >> >>> >> I am new to apex. >>> >> I tried to run demo examples in incubator-apex-malhar. >>> >> >>> >> When I ran yahoofinance example, I think it was stuck. >>> >> Log file said >>> >> 2016-05-31 20:34:00,628 INFO >>> >> com.datatorrent.bufferserver.server.Server: Server started listening >>> >> at /0:0:0:0:0:0:0:0:45546 >>> >> 2016-05-31 20:34:01,668 INFO >>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>> >> request. >>> >> >>> >> I think binding IP address has a problem. >>> >> >>> >> Does anyone help me? >>> >> >>> >> >>> >> Thanks, >>> >> Junguk >>> > >>> > >> >>
