Hi Mike, Does the RM log shows any hint?
Terence Sent from my iPhone > On May 1, 2015, at 7:35 AM, Mike Walch <[email protected]> wrote: > > I am using the CapacityScheduler. My Hadoop configuration files can be > viewed at: https://github.com/fluo-io/fluo-dev/tree/master/conf/hadoop > > While this could be a configuration issue, I don't think it's a lack of > resources as my ResourceManager has several vcores remaining after starting > my YARN app. > >> On Thu, Apr 30, 2015 at 7:57 PM Poorna Chandra <[email protected]> wrote: >> >> Hi Mike, >> >> What YARN scheduler are you using? >> >> Poorna. >> >> >>> On Thu, Apr 30, 2015 at 12:28 PM, Mike Walch <[email protected]> wrote: >>> >>> I am trying to start a Twill Runnable with 2 cores in YARN. When I set >> the >>> number of virtual cores to 2 in my ResourceSpecification, the container >>> that is started in YARN ends up having only 1 core (according to its logs >>> and the NodeManager). It looks like Twill is asking for 2 cores in the >>> ApplicationMaster but YARN only returns a container with 1 core. >>> Therefore, I am not sure if this is a Twill or YARN problem. I am >> running >>> Twill 0.5 and Hadoop 2.6.0. In my yarn-site.xml, I am not changing any >> of >>> the configuration for virtual cores from the default. Any ideas of what >>> could be causing this? >>> >>> Below are the logs from my Twill ApplicationMaster: >>> >>> 17:56:00.610 [ApplicationMasterService] INFO >>> o.a.t.i.a.ApplicationMasterService - Request 1 container with capability >>> <memory:1024, vCores:2> for runnable FluoWorker >>> >>> 17:56:02.616 [ApplicationMasterService] INFO >>> o.a.t.i.a.ApplicationMasterService - Starting runnable FluoWorker with >> RunnableProcessLauncher{container=org.apache.twill.internal.yarn.Hadoop21YarnContainerInfo@74cff77f >>> } >>> >>> I added the logging below to my Twill 0.5 branch which shows that the >>> container returned by YARN only has 1 core even though the request was >> for >>> 2: >>> >>> 17:56:02.616 [ApplicationMasterService] INFO >>> o.a.t.i.a.ApplicationMasterService - >>> processLauncher.getContainerInfo().getVirtualCores() = 1 >>> >>> 17:56:02.617 [ApplicationMasterService] INFO >>> o.a.t.i.a.ApplicationMasterService - >> provisionRequest.getRuntimeSpec().getResourceSpecification().getVirtualCores() >>> = 2 >>
