Re: Allocating Containers on a particular Node in Yarn

[gaurav]

Hi,

I had set the locality to false and request becomes as follows
/ {Priority: 0, Capability: <memory:8192, vCores:0>, # Containers: 1, 
Location: *, Relax Locality: false}, {Priority: 0, Capability: 
<memory:8192, vCores:0>, # Containers: 1, Location: /default-rack, Relax 
Locality: true}, {Priority: 0, Capability: <memory:8192, vCores:0>, # 
Containers: 1, Location: h1 Relax Locality: true}//
/
I checked AMRMClientImpl class and it is not respecting relaxlocality 
for node and racks (line 361 and368) resulting the above request.
Since one of ask has location set to h1, I expect container to be 
allocated on h1 but I get container on random hosts. I tweaked the code 
in AMRMClientImpl and made sure that relaxlocality is respected for node 
and rack, my new request becomes as following

/ {Priority: 0, Capability: <memory:8192, vCores:0>, # Containers: 1, 
Location: *, Relax Locality: false}, {Priority: 0, Capability: 
<memory:8192, vCores:0>, # Containers: 1, Location: /default-rack, Relax 
Locality: false}, {Priority: 0, Capability: <memory:8192, vCores:0>, # 
Containers: 1, Location: h1 Relax Locality: false}//
/
Now I get container on correct node h1. Not sure how will the fail over 
work in this case.

Thanks
-Gaurav

On 11/13/2013 3:23 PM, Alejandro Abdelnur wrote:
Gaurav,

Setting relaxLocality to FALSE should do it.

thanks.


On Wed, Nov 13, 2013 at 2:58 PM, gaurav <gau...@datatorrent.com> wrote:

  Hi,
  I am trying to allocate containers on a particular node in Yarn but Yarn
is returning me containers on different node although the requested node
has resources available.

I checked into the allocate(AllocateRequest request) function of
ApplicationMasterService and my request is as follows

*request: ask { priority { priority: 1 } resource_name: "h2" capability {
memory: 1000 } num_containers: 2 } ask { priority { priority: 1 }
resource_name: "/default-rack" capability { memory: 1000 } num_containers:
2 } ask { priority { priority: 1 } resource_name: "*" capability { memory:
1000 } num_containers: 2 } response_id: 1 progress: 0.0*

but the containers that I am getting back is as follows
[Container: [ContainerId: container_1384381084244_0001_01_000002, NodeId:
h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>,
Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 }, ],
Container: [ContainerId: container_1384381084244_0001_01_000003, NodeId:
h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>,
Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 }, ]]

I am attaching the test case that I have written along with the mail. It
uses classes under org.apache.hadoop.yarn.server.resourcemanager package.

Any pointers would be of great help

Thanks
Gaurav