Hi Mike, What YARN scheduler are you using?
Poorna. On Thu, Apr 30, 2015 at 12:28 PM, Mike Walch <[email protected]> wrote: > I am trying to start a Twill Runnable with 2 cores in YARN. When I set the > number of virtual cores to 2 in my ResourceSpecification, the container > that is started in YARN ends up having only 1 core (according to its logs > and the NodeManager). It looks like Twill is asking for 2 cores in the > ApplicationMaster but YARN only returns a container with 1 core. > Therefore, I am not sure if this is a Twill or YARN problem. I am running > Twill 0.5 and Hadoop 2.6.0. In my yarn-site.xml, I am not changing any of > the configuration for virtual cores from the default. Any ideas of what > could be causing this? > > Below are the logs from my Twill ApplicationMaster: > > 17:56:00.610 [ApplicationMasterService] INFO > o.a.t.i.a.ApplicationMasterService - Request 1 container with capability > <memory:1024, vCores:2> for runnable FluoWorker > > 17:56:02.616 [ApplicationMasterService] INFO > o.a.t.i.a.ApplicationMasterService - Starting runnable FluoWorker with > > RunnableProcessLauncher{container=org.apache.twill.internal.yarn.Hadoop21YarnContainerInfo@74cff77f > } > > I added the logging below to my Twill 0.5 branch which shows that the > container returned by YARN only has 1 core even though the request was for > 2: > > 17:56:02.616 [ApplicationMasterService] INFO > o.a.t.i.a.ApplicationMasterService - > processLauncher.getContainerInfo().getVirtualCores() = 1 > > 17:56:02.617 [ApplicationMasterService] INFO > o.a.t.i.a.ApplicationMasterService - > > provisionRequest.getRuntimeSpec().getResourceSpecification().getVirtualCores() > = 2 >
