[
https://issues.apache.org/jira/browse/YARN-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705283#comment-13705283
]
Bikas Saha commented on YARN-521:
---------------------------------
Shouldnt this also check that the inferred rack1 has relaxLocality set to false?
{code}
+ ContainerRequest bothLevelRequest =
+ new ContainerRequest(capability, new String[] {"host3", "host4"},
+ new String[] {NetworkTopology.DEFAULT_RACK, "/otherrack"},
+ Priority.newInstance(3), 4, false);
+ client.addContainerRequest(bothLevelRequest);
+
+ verifyResourceRequest(client, bothLevelRequest, ResourceRequest.ANY,
false);
+ verifyResourceRequest(client, bothLevelRequest,
NetworkTopology.DEFAULT_RACK,
+ true);
+ verifyResourceRequest(client, bothLevelRequest, "/otherrack",
+ true);
+ verifyResourceRequest(client, bothLevelRequest, "host3", true);
+ verifyResourceRequest(client, bothLevelRequest, "host4", true);
+
{code}
The RM allows relaxLocality to be re-enabled on a location and priority. This
allows users to change their minds about strict locality if they are unable to
get those containers for a long time. The logic in the patch does not allow
this to happen. In fact testDifferentLocalityRelaxationSamePriority() shows
that it is not possible to do this with the current patch.
testDifferentLocalityRelaxationSamePriority() looks to have been written to
test for the case where host1 is specific resulting in rack1 to be false. Then
host3 in rack1 cannot be made non-specific since the flag on rack1 will be
inconsistent. It would be more clear if the second request had a different host.
testLocalityRelaxationDifferentLevels() tests that specific host cannot be
mixed with non-specific rack for the rack of that host. Can specific host be
mixed with specific rack for the same rack as the host? Probably not.
Can specific node host1 be mixed with specific rack on a different rack
(rack2)? If yes, if a container is allocated on host2 in rack2, then what
prevents getMachingRequests(*) from returning the requests for host1 and rack2
instead of only the request for rack2?
Now that we have a client and server pieces done, has this been tested with a
real cluster to see that it works for in practice?
It would really help if we had a clear document (even javadoc) that clarifies
what is legal and what is not. This will really help in code review, it will
make sure we have better test coverage and in general will help users
understand how to use the API. I am concerned that we are implementing
something pretty subtle and we need to get it right logically before
implementing the code. I havent even started thinking about how this interacts
with the blacklisting logic that we have added to the RM. How specific requests
interact with blacklist on the RM. And how both interact on the AMRMClient. And
how both interact across RM and AMRMClient.
Minor nits.
How about "inferredRacks" instead of "filledInRacks"?
How about JAVA standard library Collections.singletonList() instead of invoking
google common Lists.newArrayList?
> Augment AM - RM client module to be able to request containers only at
> specific locations
> -----------------------------------------------------------------------------------------
>
> Key: YARN-521
> URL: https://issues.apache.org/jira/browse/YARN-521
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api
> Affects Versions: 2.0.3-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: YARN-521-1.patch, YARN-521-2.patch, YARN-521-2.patch,
> YARN-521.patch
>
>
> When YARN-392 and YARN-398 are completed, it would be good for AMRMClient to
> offer an easy way to access their functionality
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
