[
https://issues.apache.org/jira/browse/YARN-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705341#comment-13705341
]
Sandy Ryza commented on YARN-521:
---------------------------------
bq. Shouldnt this also check that the inferred rack1 has relaxLocality set to
false?
This was meant to test the case where the inferred rack is also explicitly
specified. Looks like I messed up NetworkTopology.DEFAULT_RACK vs. the one
that MyResolver returns. Will fix this.
bq. It would be more clear if the second request had a different host.
Will make this change.
bq. testLocalityRelaxationDifferentLevels() tests that specific host cannot be
mixed with non-specific rack for the rack of that host. Can specific host be
mixed with specific rack for the same rack as the host? Probably not.
I meant for the documentation to cover this case: "If true, containers for this
request may be assigned on hosts and racks other than the ones explicitly
requested.", meaning that a container may be placed on any host on the rack,
and that the host is redundant. I am OK with throwing an exception in this
case if you think that is better.
bq. Can specific node host1 be mixed with specific rack on a different rack
(rack2)? If yes, if a container is allocated on host2 in rack2, then what
prevents getMachingRequests from returning the requests for host1 and rack2
instead of only the request for rack2?
The way I see it, getMatchingRequests(priority1, * ) should give me all the
requests at priority1, independent of locality relaxation.
bq. Now that we have a client and server pieces done, has this been tested with
a real cluster to see that it works for in practice?
I haven't done this and won't have a chance in the next couple days. If
possible I would like to get this into 2.1.0-beta as there have been recent
requests on the Hadoop user list for it, and including YARN-392 but omitting
this will lead them away from AMRMClient (ref:
http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201307.mbox/%3ccahg+sbmenvpsp4t5tbl3mi1osrjk-6+nxrafeko4ldelhps...@mail.gmail.com%3E).
bq. I am concerned that we are implementing something pretty subtle and we need
to get it right logically before implementing the code.
I understand the concern and agree that some of the cases are pretty subtle. I
have thought about this quite a bit and am convinced that the current approach
is as coherent as we can get within the resource request model. I will try to
lay it out more clearly in the javadoc and copy it on the JIRA to make it
easier for posterity to reference. Ideally we would add it to the "Writing
YARN Applications" documentation, but as it is very out of date and does not
even reference AMRMClient, I think that is out of the scope of this JIRA.
bq. The RM allows relaxLocality to be re-enabled on a location and priority.
This allows users to change their minds about strict locality if they are
unable to get those contabqiners for a long time. The logic in the patch does
not allow this to happen. In fact testDifferentLocalityRelaxationSamePriority()
shows that it is not possible to do this with the current patch.
My intended behavior is that to do this the user must remove the original
container request before adding a new one with different locality relaxation.
Am I misunderstanding how removeContainerRequest works again?
bq. Minor nits.
Will make these changes.
> Augment AM - RM client module to be able to request containers only at
> specific locations
> -----------------------------------------------------------------------------------------
>
> Key: YARN-521
> URL: https://issues.apache.org/jira/browse/YARN-521
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api
> Affects Versions: 2.0.3-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: YARN-521-1.patch, YARN-521-2.patch, YARN-521-2.patch,
> YARN-521.patch
>
>
> When YARN-392 and YARN-398 are completed, it would be good for AMRMClient to
> offer an easy way to access their functionality
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
