I am using the CapacityScheduler. My Hadoop configuration files can be viewed at: https://github.com/fluo-io/fluo-dev/tree/master/conf/hadoop
While this could be a configuration issue, I don't think it's a lack of resources as my ResourceManager has several vcores remaining after starting my YARN app. On Thu, Apr 30, 2015 at 7:57 PM Poorna Chandra <[email protected]> wrote: > Hi Mike, > > What YARN scheduler are you using? > > Poorna. > > > On Thu, Apr 30, 2015 at 12:28 PM, Mike Walch <[email protected]> wrote: > > > I am trying to start a Twill Runnable with 2 cores in YARN. When I set > the > > number of virtual cores to 2 in my ResourceSpecification, the container > > that is started in YARN ends up having only 1 core (according to its logs > > and the NodeManager). It looks like Twill is asking for 2 cores in the > > ApplicationMaster but YARN only returns a container with 1 core. > > Therefore, I am not sure if this is a Twill or YARN problem. I am > running > > Twill 0.5 and Hadoop 2.6.0. In my yarn-site.xml, I am not changing any > of > > the configuration for virtual cores from the default. Any ideas of what > > could be causing this? > > > > Below are the logs from my Twill ApplicationMaster: > > > > 17:56:00.610 [ApplicationMasterService] INFO > > o.a.t.i.a.ApplicationMasterService - Request 1 container with capability > > <memory:1024, vCores:2> for runnable FluoWorker > > > > 17:56:02.616 [ApplicationMasterService] INFO > > o.a.t.i.a.ApplicationMasterService - Starting runnable FluoWorker with > > > > > RunnableProcessLauncher{container=org.apache.twill.internal.yarn.Hadoop21YarnContainerInfo@74cff77f > > } > > > > I added the logging below to my Twill 0.5 branch which shows that the > > container returned by YARN only has 1 core even though the request was > for > > 2: > > > > 17:56:02.616 [ApplicationMasterService] INFO > > o.a.t.i.a.ApplicationMasterService - > > processLauncher.getContainerInfo().getVirtualCores() = 1 > > > > 17:56:02.617 [ApplicationMasterService] INFO > > o.a.t.i.a.ApplicationMasterService - > > > > > provisionRequest.getRuntimeSpec().getResourceSpecification().getVirtualCores() > > = 2 > > >
