> On 12 Jun 2015, at 02:10, Navina Ramesh <[email protected]> wrote: > > Hi yarn-devs, > > I am trying to utilize YARN’s host-affinity feature in Apache Samza > (SAMZA-617). > > I have a question regarding the order in which the resource requests are > executed by the RM. Today, we make successive requests for x (say x = 3) > containers for deploying a Samza job. > It looks like: > Request 1: (container-0, $hostX) > Request 2: (container-1, $hostY) > Request 3: (container-2, $hostZ)
does it when space comes up. > > When implementing the callback “onContainerAllocated” in the SamzaAppMaster, > how can I associate an allocated container with its corresponding > containerRequest? Is there a way in YARN to associate an allocated container > to its request? > > If not, is it correct to assume > that the RM handles the requests in a FIFO manner and hence, the order in > which the onContainerAllocated callback is invoked will be the same as the > request order? > that's a bad strategy. What if hostY had capacity before hostX? Would you expect it to be blocked until hostX was satisifed? as hostY may not be free then. > This information will be very useful for Samza to implement host-affinity in > its deployment model. Please let me know. > If you are explicilty requesting nodes on specific hosts, with relaxLocality==false, then the responses you get back will be for the hosts you asked for. All you need is a map of (hostname->request) to look them back up, provided you have exactly one request per host outstanding. For Apache Slider (incubating) we use a different YARN priority for different component types; each, so can have >1 explicit request for a host provided they are for different components. What we can't do is tell which request with relaxLocality=true has been satisfied when a response comes back for a host that wasn't explicly asked for. For example, if you get got an allocation for a container on host W, which one of the other requests is no longer outstanding?
