[
https://issues.apache.org/jira/browse/YARN-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472863#comment-13472863
]
Siddharth Seth commented on YARN-103:
-------------------------------------
bq. This client is the barebones implementation and hence has no smarts. The
checks can be implemented/may not be necessary in smarter versions of the
client. Additionally, like you say, some information may be better exposed at
different API's.
Why have an additional smarter client on top of this one in YARN ? Individual
apps may choose to have their own - MR for RPC specific error handling, etc. We
could even considering getting rid of getNumClusterNodes /
getAvailableResources since they're available via AllocateResponse.
Would like others to take a look at the entire API specification before
committing this.
bq. I actually want to test against the MiniMRCluster instead of an expected
behavior via a mockRM. I want the client test to fail if the RM behavior
changes. So yes this is more of a functional test than a unit test.
RM behaviour depends upon the underlying scheduler. For a simple case like
this, the behaviour is likely to be the same across schedulers - but the test
runs against the default - which is the CS right now. We don't differentiate
between functional / unit tests at the moment - which makes the test runtimes
reasonably high. (MiniClusetr tests take several seconds versus fractions of a
second for mocks).
In the current functional test - there's a sleep-poll cycle which could get
stuck.
bq. Per your comments in MAPREDUCE-4671, I have attached another version of the
4th patch that uses a Wrapper class for ResourceRequest instead of not deleting
objects. You can compare both and see which one seems more palatable. Based on
that MAPREDUCE-4671 can be updated if needed.
A custom comparator may be sufficient - o.a.h.y.s.r.Application already uses
this. Alternately, using the wrapper only for the 'ask' table may be simpler ?
Looks like the patches accidentally include some hdfs changes.
> Add a yarn AM - RM client module
> --------------------------------
>
> Key: YARN-103
> URL: https://issues.apache.org/jira/browse/YARN-103
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: YARN-103.1.patch, YARN-103.2.patch, YARN-103.3.patch,
> YARN-103.4.patch, YARN-103.4.wrapper.patch
>
>
> Add a basic client wrapper library to the AM RM protocol in order to prevent
> proliferation of code being duplicated everywhere. Provide helper functions
> to perform reverse mapping of container requests to RM allocation resource
> request table format.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
