> On 12 Jun 2015, at 20:20, Navina Ramesh <[email protected]> wrote: > > Hi Steve, > Thanks for your response. > > I think I should have mentioned a couple of design choices we made: > 1. *Continuous Scheduling* - I explored the option of requesting > containers with relaxLocality==false. But this will add to latency and > there is no way of revoking the resource request.
You can cancel requests. AMRMClientAsync.removeContainerRequest(request); > An alternate approach > was to use continuous scheduling with the FairScheduler. Instead of > waiting on node heartbeats, the RM iterates through the nodes based on the > latest known states. We enable this and also, associate a rack and node > level delay. This way if the requested resource is not available, it will > automatically relaxLocality and return any available resource. > > 2. All the resource request that I make is for running the same component > and hence, their priority is the same. > >>> that's a bad strategy. What if hostY had capacity before hostX? Would >>> you expect it to be blocked until hostX was satisifed? as hostY may not >>> be free then > It is fine if the hostY gets allocated before hostX. I do maintain a map > of resource request to requested host. So, if hostY is returned, I will > know that I expected that host for container-1. The issue is how to > associate an allocated resource that is not in my list of preferred hosts. > If a random hostA is allocated, then which container should I assign ? > This is where I am stuck and I am wondering if there is a way around. > I couldn't find a way here. When things come in to an unexpected host they just get an instance of that component type assigned to it. that leaves outstanding requests in the list, I deal with that when eventually all the containers get satisified -at that point I know everything outstanding is satisified, so cancel all the requests. I'd have liked a request ID to do the mapping, but that would apparently make aggregating requests impossible, so no.
