Just to be sure, did that change resolve the issue ? Ram
On Fri, Dec 2, 2016 at 11:54 AM, Max Bridgewater <max.bridgewa...@gmail.com> wrote: > OK. Thanks guys. It continued to fail with min 256 and max 512 or 1024. In > the end I switched off the check of ratio virtual/physical memory. My > config is now: > > <property> > <name>yarn.nodemanager.vmem-check-enabled</name> > <value>false</value> > </property> > <property> > <name>yarn.scheduler.minimum-allocation-mb</name> > <value>256</value> > </property> > <property> > <name>yarn.scheduler.maximum-allocation-mb</name> > <value>1024</value> > </property> > > > > > On Fri, Dec 2, 2016 at 2:06 PM, AJAY GUPTA <ajaygit...@gmail.com> wrote: > >> Hi Max/Ashwin >> >> The logs have the following lines >> 2016-12-02 16:53:53,868 INFO com.datatorrent.stram.StreamingAppMasterService: >> Asking RM for containers: [Capability[<memory:3000, vCores:1>]Priority[2], >> Capability[<memory:3000, vCores:1>]Priority[1]] >> 2016-12-02 16:53:53,868 INFO com.datatorrent.stram.StreamingAppMasterService: >> Requested container: Capability[<memory:3000, vCores:1>]Priority[2] on >> host: [null] >> >> The yarn manager has been requested for 3000MB RAM which the system is >> probably not able to provide. >> >> With the above configuration, you have set a minimum allocation of 256MB >> per operator and maximum of 512MB per operator. You can change this later >> in case you expect your operator to consume more/less memory. >> >> Ashwin, kindly correct me if I am wrong here. >> >> >> Regards, >> Ajay >> >> On Sat, Dec 3, 2016 at 12:28 AM, Ashwin Chandra Putta < >> ashwinchand...@gmail.com> wrote: >> >>> Ajay, >>> >>> Can you specify the reason why it did not work before as well? >>> >>> Regards, >>> Ashwin. >>> >>> On Fri, Dec 2, 2016 at 10:55 AM, AJAY GUPTA <ajaygit...@gmail.com> >>> wrote: >>> >>>> Hi Max, >>>> >>>> Can you try adding the following configurations to yarn-site.xml. >>>> Restart yarn and then try starting wordcount-demo. >>>> >>>> >>>> <property> >>>> <name>yarn.scheduler.minimum-allocation-mb</name> >>>> <value>256</value> >>>> </property> >>>> <property> >>>> <name>yarn.scheduler.maximum-allocation-mb</name> >>>> <value>512</value> >>>> </property> >>>> >>>> >>>> Regards, >>>> Ajay >>>> >>>> >>>> On Fri, Dec 2, 2016 at 10:26 PM, Max Bridgewater < >>>> max.bridgewa...@gmail.com> wrote: >>>> >>>>> Yeah, application has been running for 20h. But no event is flowing >>>>> through. See dt.log attached. >>>>> >>>>> On Fri, Dec 2, 2016 at 11:48 AM, Ashwin Chandra Putta < >>>>> ashwinchand...@gmail.com> wrote: >>>>> >>>>>> Max, >>>>>> >>>>>> Can you check app master logs? If application status changed to >>>>>> running, it means app master is running. You can find operator deployment >>>>>> related logs from app master dt.log. >>>>>> >>>>>> Regards, >>>>>> Ashwin. >>>>>> >>>>>> On Dec 2, 2016 5:25 AM, "Max Bridgewater" <max.bridgewa...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I deployed the WordCountDemo in DataTorrent RTS Enterprise with >>>>>>> evaluation license. The application is in RUNNING state and resulted >>>>>>> into >>>>>>> two processes that are all in ACTIVE state. >>>>>>> >>>>>>> On the other hand, however, the operators themselves are in >>>>>>> PENDING_DEPLOY state. These are wordinput, count, and console. So, >>>>>>> nothing >>>>>>> is really running and no words are being counted. >>>>>>> >>>>>>> There seems to be enough resources: >>>>>>> >>>>>>> 2016-12-01 20:36:28,287 INFO org.apache.hadoop.yarn.server. >>>>>>> resourcemanager.sched >>>>>>> uler.SchedulerNode: Assigned container >>>>>>> container_1480549373717_0021_01_000002 >>>>>>> of >>>>>>> capacity <memory:3000, vCores:1> on host localhost:36079, which has >>>>>>> 2 container >>>>>>> s, <memory:6000, vCores:2> used and <memory:2192, vCores:6> >>>>>>> available after allo >>>>>>> cation. >>>>>>> >>>>>>> >>>>>>> In /var/log/dtgateway.log, here is what I see. Can this be the >>>>>>> cause? If so, how do I fix this? >>>>>>> >>>>>>> 016-12-02 13:21:34,113 WARN com.datatorrent.gateway.x: Cannot update >>>>>>> license registry for the number of nodes >>>>>>> com.datatorrent.a.E: Filesystem closed >>>>>>> at com.datatorrent.a.M.b(w:341) >>>>>>> at com.datatorrent.a.C.b(m:34) >>>>>>> at com.datatorrent.gateway.x.b(jd:456) >>>>>>> at com.datatorrent.gateway.x.b(jd:627) >>>>>>> at com.datatorrent.gateway.x.b(jd:787) >>>>>>> at com.datatorrent.gateway.x.b(jd:141) >>>>>>> at com.datatorrent.gateway.U.run(jd:210) >>>>>>> at java.util.concurrent.Executors >>>>>>> $RunnableAdapter.call(Executors.java:51 >>>>>>> 1) >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>>>>> at java.util.concurrent.ThreadPoo >>>>>>> lExecutor.runWorker(ThreadPoolExecutor. >>>>>>> java:1142) >>>>>>> at java.util.concurrent.ThreadPoo >>>>>>> lExecutor$Worker.run(ThreadPoolExecutor >>>>>>> .java:617) >>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>> 2016-12-02 13:21:34,113 WARN com.datatorrent.gateway.x: Detected >>>>>>> containers not >>>>>>> provisioned for application_1480549373717_0021: # planned >>>>>>> containers is 3 and # >>>>>>> allocated containers is 1 >>>>>>> 2016-12-02 13:21:39,158 WARN com.datatorrent.gateway.x: Cannot >>>>>>> update license re >>>>>>> gistry for the number of nodes >>>>>>> com.datatorrent.a.E: Filesystem closed >>>>>>> at com.datatorrent.a.M.b(w:341) >>>>>>> at com.datatorrent.a.C.b(m:34) >>>>>>> at com.datatorrent.gateway.x.b(jd:456) >>>>>>> at com.datatorrent.gateway.x.b(jd:627) >>>>>>> at com.datatorrent.gateway.x.b(jd:787) >>>>>>> at com.datatorrent.gateway.x.b(jd:141) >>>>>>> at com.datatorrent.gateway.U.run(jd:210) >>>>>>> at java.util.concurrent.Executors >>>>>>> $RunnableAdapter.call(Executors.java:51 >>>>>>> 1) >>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >>> >>> >>> -- >>> >>> Regards, >>> Ashwin. >>> >> >> >
