Hi. Sorry for many emails. However, I think something is wrong. It continuously created log files container_1464766676472_0003_01_xxxx, which meant java process keep dead.
Is it right? Was it supposed to do? Thanks in advance. Junguk 2016-06-01 1:57 GMT-06:00 Junguk Cho <[email protected]>: > Hi, Tushar. > > Thanks. > Yes. It worked. > > I have one more question. > When I access to hadoop web UI and there is menu "ApplicationMaster" > in Tracking UI menu. > When I click this, it gave me error. > > Sorry, got error 404 > > Please consult RFC 2616 for meanings of the error code. > > Error Details > > org.apache.hadoop.yarn.webapp.WebAppException: /: controller for > default not found > at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232) > at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140) > at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263) > at > com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178) > at > com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) > at > com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62) > at > com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900) > at > com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834) > at > com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795) > at > com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) > at > com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) > at > com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) > at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) > at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) > at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) > at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) > > The URL is "http://<IP address>:8088/proxy/application_1464766676472_0002/. > > With this menu, what we can see? > > > Thanks, > Junguk > > 2016-06-01 1:24 GMT-06:00 Tushar Gosavi <[email protected]>: >> Hi Junguk, >> >> Please check your yarn configuration settings. I normally set >> minimum-allocation-mb to 256MB for single node install, you can add >> following lines to your yarn-site.xml and restart the yarn. >> >> <property> >> <name>yarn.scheduler.minimum-allocation-mb</name> >> <value>256</value> >> </property> >> >> I am able to run YahooFinanceDemo app with above setting on my laptop >> with 5GB memory usage. >> >> Regards, >> -Tushar. >> >> On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh >> <[email protected]> wrote: >>> Thanks for details. >>> >>> Typically the "Waiting for pending status" is followed by "Deploy request" >>> as highlighted below. Are you getting the highlighted line in your log? If >>> not, the operators is not getting deployed correctly. Please let us know. >>> >>> >>> 2016-05-23 14:46:51,649 DEBUG >>> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512 >>> 2016-05-23 14:46:51,655 INFO >>> com.datatorrent.bufferserver.storage.DiskStorage: using >>> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp >>> as the basepath for spooling. >>> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server: >>> Server started listening at /0:0:0:0:0:0:0:0:42358 >>> 2016-05-23 14:46:51,665 DEBUG >>> com.datatorrent.stram.engine.StreamingContainer: Buffer server started: >>> /0:0:0:0:0:0:0:0:42358 >>> 2016-05-23 14:46:51,700 DEBUG >>> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop >>> (interval is 1000 ms) >>> 2016-05-23 14:46:52,750 INFO >>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>> request. >>> 2016-05-23 14:46:53,255 INFO >>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>> request. >>> 2016-05-23 14:46:56,287 INFO >>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>> request. >>> 2016-05-23 14:46:56,791 INFO >>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>> request. >>> 2016-05-23 14:46:57,295 INFO >>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>> request. >>> 2016-05-23 14:46:57,809 INFO >>> com.datatorrent.stram.engine.StreamingContainer: Deploy request: >>> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff, >>> 0, >>> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]] >>> 2016-05-23 14:46:57,811 DEBUG >>> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to >>> checkpoint ffffffffffffffff stateless=false. >>> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT, >>> operator class=class com.datatorrent.InputFileReader >>> 2016-05-23 14:46:57,889 DEBUG >>> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as >>> deployed. >>> >>> >>> Regards, >>> Sandeep >>> >>> Regards, >>> Sandeep >>> >>> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <[email protected]> wrote: >>>> >>>> Hi, >>>> >>>> 1. I used only one machine and use docker image to run it. >>>> When I access to ip:8088, it shows I use 8 GB memory. >>>> (Is there a configuration file I can check?) >>>> >>>> 2. Hadoop version : 2.7.1 >>>> I tested both Apex 3.2.2 and Apex 3.3. >>>> >>>> Thanks, >>>> Junguk >>>> >>>> >>>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <[email protected]>: >>>> > Hi Junguk, >>>> > >>>> > Could you please give us some details: >>>> > >>>> > Hadoop cluster configuration: no of machines and their configuration >>>> > (memory >>>> > in particular) >>>> > Version of Hadoop and Apex >>>> > >>>> > Regards, >>>> > Sandeep >>>> > >>>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <[email protected]> wrote: >>>> >> >>>> >> Hi, >>>> >> >>>> >> I am new to apex. >>>> >> I tried to run demo examples in incubator-apex-malhar. >>>> >> >>>> >> When I ran yahoofinance example, I think it was stuck. >>>> >> Log file said >>>> >> 2016-05-31 20:34:00,628 INFO >>>> >> com.datatorrent.bufferserver.server.Server: Server started listening >>>> >> at /0:0:0:0:0:0:0:0:45546 >>>> >> 2016-05-31 20:34:01,668 INFO >>>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending >>>> >> request. >>>> >> >>>> >> I think binding IP address has a problem. >>>> >> >>>> >> Does anyone help me? >>>> >> >>>> >> >>>> >> Thanks, >>>> >> Junguk >>>> > >>>> > >>> >>>
