[
https://issues.apache.org/jira/browse/YARN-660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13655049#comment-13655049
]
Bikas Saha commented on YARN-660:
---------------------------------
Vinod, I have iterated through different combinations of generics including the
one you suggest. I found that the current version is the least ugly wrt user
code. Both the test code for the AMRMClient as well as my use of it in TEZ gave
me that impression. If you write user code that has to always cast/convert back
from ContainerRequest to user-defined types you will see that it looks ugly and
I dont want users to keep having to cast these value when they use specific
types. The only user overhead when they use ContainerRequest is to add a
<ContainerRequest> when declaring the member variable and during new.
Thereafter everything works. And ContainerRequest is something that they
anyways have to include in the code. Look at the diff of TestAMRMClient to see
the minimal changes needed to make the existing code compile. Ideally if there
was a typedef in Java then even that could be hidden but there isnt.
IMO, ContainerRequest is useful mostly in cases when the user wants a bunch of
containers at *. For most of the other cases where requests are made with more
specificity StoredContainerRequest is more useful. Its provided by the library
because it will be commonly needed and also to support returning matching
requests from within the AMRMClient for reasons outlined earlier. Storing and
retrieving matching requests in a meaningful manner cannot be done until we
limit the number of containers in an individual request to 1.
StoredContainerRequest provides a clear type inside the AMRMClient to say what
to store and also to limit the container count per request to 1. e.g. if
AMRMClient will save simple ContainerRequest then lets say it saves the
container request in addContainerRequest(ContainerRequest(P1,R1,Count=4)) and
then if user calls removeContainerRequest(ContainerRequest(P1,R1,Count=3)) its
hard for AMRMClient to tell if the stored container request should be removed
or not.
Cookies are not required but very useful in reducing user burden. If we think
about using AMRMClient inside the MR app master (or see the impl of TEZ) then
we will find that for every request we need to save a cookie somewhere (eg.
Scheduler*LaunchEvent) that will be used when the request is matched to a
container. Either the client can write a map to maintain and store this
relationship or the library provides a helper cookie to keep the info in one
place.
Let me know if you have any other comments.
> Improve AMRMClient with cookies and matching requests
> -----------------------------------------------------
>
> Key: YARN-660
> URL: https://issues.apache.org/jira/browse/YARN-660
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: 2.0.5-beta
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Fix For: 2.0.5-beta
>
> Attachments: YARN-660.1.patch, YARN-660.2.patch, YARN-660.3.patch,
> YARN-660.4.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
