Requesting the rack is not necessary, and is leading to the behavior that you're seeing.
The documentation states: * <li>If locality relaxation is disabled, then only within the same request, * a node and its rack may be specified together. This allows for a specific * rack with a preference for a specific node within that rack.</li> On Wed, Nov 13, 2013 at 10:10 PM, gaurav <[email protected]> wrote: > Here is the snippet of code that I am using to allocate containers > > AMRMClient<ContainerRequest> amRmClient = > AMRMClient.createAMRMClient();; > String host = "h1"; > Resource capability = Records.newRecord(Resource.class); > capability.setMemory(memory); > nodes = new String[] {host}; > // in order to request a host, we also have to request the rack > racks = new String[] {"/default-rack"}; > List<ContainerRequest> containerRequests = new > ArrayList<ContainerRequest>(); > List<ContainerId> releasedContainers = new ArrayList<ContainerId>(); > containerRequests.add(new ContainerRequest(capability, nodes, racks, > Priority.newInstance(priority),false)); > if (containerRequests.size() > 0) { > LOG.info("Asking RM for containers: " + containerRequests); > for (ContainerRequest cr : containerRequests) { > LOG.info("Requested container: {}", cr.toString()); > amRmClient.addContainerRequest(cr); > } > } > > for (ContainerId containerId : releasedContainers) { > LOG.info("Released container, id={}", containerId.getId()); > amRmClient.releaseAssignedContainer(containerId); > } > return amRmClient.allocate(0); > > > > -Gaurav > > > On 11/13/2013 07:36 PM, Sandy Ryza wrote: > > In that case, the AMRMClient code looks correct to me. Can you share the > code you've written against it that's not receiving the correct containers? > > > On Wed, Nov 13, 2013 at 5:30 PM, gaurav <[email protected]> wrote: > >> Hi, >> >> I looked in the trunk and line numbers are 361 and 366. >> >> Thanks >> -Gaurav >> >> On 11/13/2013 5:04 PM, gaurav wrote: >> >> I have hadoop-2.2.0 >> >> Thanks >> -Gaurav >> >> On 11/13/2013 4:59 PM, Sandy Ryza wrote: >> >> What version are you using? Setting the relax locality to true for nodes >> is always correct. For racks, this is not necessarily the case. When I >> look at trunk, it is set to true always on line 361 (which is correct), but >> on on line 374. >> >> -Sandy >> >> >> On Wed, Nov 13, 2013 at 4:47 PM, gaurav <[email protected]> wrote: >> >>> Hi Sandy, >>> >>> No it is not working for me. As mentioned earlier, AMRMClient is not >>> respecting the locality for node and rack in *Line 361 and 374 *and is >>> set to true always. >>> I am requesting for one container with specified node and rack and >>> relaxed locality false. >>> >>> Thanks >>> -Gaurav >>> >>> On 11/13/2013 4:24 PM, Sandy Ryza wrote: >>> >>> [moving to user list] >>> >>> Right. relaxLocality needs to be set on the next level up. It >>> determines whether locality can be relaxed to that level. Confusing, I >>> know. If you are using AMRMClient, you should be able to accomplish what >>> you're looking for by creating an AMRMClient.ContainerRequest that >>> specifies a node and sets relax locality to false. Is that not working for >>> you? >>> >>> -Sandy >>> >>> >>> On Wed, Nov 13, 2013 at 4:19 PM, gaurav <[email protected]> wrote: >>> >>>> Hi Sandy, >>>> >>>> If I set relaxlocality to true with host name, I don't get the >>>> container on specified host even though node has the resources. >>>> I am using AMRMClient, only thing is I made following changes to get >>>> the containers on the specified node. >>>> / >>>> //Line: 361 // >>>> //Original// >>>> // addResourceRequest(req.getPriority(), node, req.getCapability(), >>>> req, true);// >>>> //Modifiled// >>>> // addResourceRequest(req.getPriority(), node, req.getCapability(), >>>> req, req.getRelaxLocality());// >>>> // >>>> //Line 374// >>>> //Original// >>>> // addResourceRequest(req.getPriority(), rack, req.getCapability(), >>>> req, true);// >>>> //Modifiled// >>>> // addResourceRequest(req.getPriority(), rack, req.getCapability(), >>>> req, req.getRelaxLocality());// >>>> /// >>>> >>>> >>>> Thanks >>>> -Gaurav >>>> >>>> >>>> On 11/13/2013 4:02 PM, Sandy Ryza wrote: >>>> >>>>> Yeah, specifying a host name with relaxLocality is meaningful. >>>>> Schedulers >>>>> use delay scheduling ( >>>>> http://www.cs.berkeley.edu/~matei/talks/2010/eurosys_delaysched.pdf) >>>>> to >>>>> achieve locality when relaxLocality is on. But it is turned off by >>>>> default. The individual scheduler docs have specifics on how to >>>>> configure >>>>> it. >>>>> >>>>> Guarav, >>>>> Using ResourceRequests directly is not straightforward and error >>>>> prone. Is >>>>> there a reason that AMRMClient is unsuitable for your needs? >>>>> >>>>> -Sandy >>>>> >>>>> >>>>> >>>>> On Wed, Nov 13, 2013 at 3:55 PM, Thomas Weise <[email protected] >>>>> >wrote: >>>>> >>>>> Is it possible to specify a particular node and have RM fallback to an >>>>>> different node only after making an attempt to allocate for the >>>>>> requested >>>>>> node? In other words, is the combination of specific host name and >>>>>> relaxLocality=TRUE meaningful at all? >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>> On Wed, Nov 13, 2013 at 3:23 PM, Alejandro Abdelnur < >>>>>> [email protected] >>>>>> >>>>>>> wrote: >>>>>>> Gaurav, >>>>>>> >>>>>>> Setting relaxLocality to FALSE should do it. >>>>>>> >>>>>>> thanks. >>>>>>> >>>>>>> >>>>>>> On Wed, Nov 13, 2013 at 2:58 PM, gaurav <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>>> I am trying to allocate containers on a particular node in Yarn >>>>>>>> but >>>>>>>> >>>>>>> Yarn >>>>>> >>>>>>> is returning me containers on different node although the requested >>>>>>>> >>>>>>> node >>>>>> >>>>>>> has resources available. >>>>>>>> >>>>>>>> I checked into the allocate(AllocateRequest request) function of >>>>>>>> ApplicationMasterService and my request is as follows >>>>>>>> >>>>>>>> *request: ask { priority { priority: 1 } resource_name: "h2" >>>>>>>> >>>>>>> capability { >>>>>> >>>>>>> memory: 1000 } num_containers: 2 } ask { priority { priority: 1 } >>>>>>>> resource_name: "/default-rack" capability { memory: 1000 } >>>>>>>> >>>>>>> num_containers: >>>>>>> >>>>>>>> 2 } ask { priority { priority: 1 } resource_name: "*" capability { >>>>>>>> >>>>>>> memory: >>>>>>> >>>>>>>> 1000 } num_containers: 2 } response_id: 1 progress: 0.0* >>>>>>>> >>>>>>>> but the containers that I am getting back is as follows >>>>>>>> [Container: [ContainerId: container_1384381084244_0001_01_000002, >>>>>>>> >>>>>>> NodeId: >>>>>> >>>>>>> h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>, >>>>>>>> Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 >>>>>>>> }, >>>>>>>> >>>>>>> ], >>>>>> >>>>>>> Container: [ContainerId: container_1384381084244_0001_01_000003, >>>>>>>> >>>>>>> NodeId: >>>>>> >>>>>>> h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>, >>>>>>>> Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 >>>>>>>> }, >>>>>>>> >>>>>>> ]] >>>>>> >>>>>>> I am attaching the test case that I have written along with the >>>>>>>> mail. >>>>>>>> >>>>>>> It >>>>>> >>>>>>> uses classes under org.apache.hadoop.yarn.server.resourcemanager >>>>>>>> >>>>>>> package. >>>>>> >>>>>>> Any pointers would be of great help >>>>>>>> >>>>>>>> Thanks >>>>>>>> Gaurav >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> Alejandro >>>>>>> >>>>>>> >>>> >>> >>> >> >> >> > >
