[
https://issues.apache.org/jira/browse/YARN-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13657452#comment-13657452
]
Sandy Ryza commented on YARN-664:
---------------------------------
My mistake - the original example was valid, just switched it to be an invalid
one.
The AMRM protocol expects a corresponding rack-level and *-level request for
every node-level request, and a corresponding *-level request for every
rack-level request. The requests in the updated example now mean something
like "if you give me a container on rack1, it should have 2048 MB, but if it's
on any other rack, it should have 1024 MB". I don't think the scheduler should
support this, as it's hard to imagine when it would be necessary, and makes it
impossible to calculate an application's demand.
If what you're saying is that the scheduler should be able to support requests
like:
priority=1, location=*, memory=1024
priority=1, location=rack1, memory=1024
priority=1, location=*, memory=2048
priority=1, location=rack1, memory=2048
that might make sense. I had filed YARN-314 for this a while ago, but have
since become less convinced of its utility. It would require deepening the
data structures in the scheduler, which would mean extra hash lookups for each
request. The philosophy I've perceived in the design has been that containers
with different requirements should be requested at explicitly different
priorities. [~acm], would you be able to weigh in?
> throw InvalidRequestException for requests with different capabilities at the
> same priority
> -------------------------------------------------------------------------------------------
>
> Key: YARN-664
> URL: https://issues.apache.org/jira/browse/YARN-664
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, scheduler
> Affects Versions: 2.0.4-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
>
> Nothing stops an application from submitting a request with priority=1,
> location=*, memory=1024 and a request with priority=1, location=rack1,
> memory=2048. However, this does not make sense under the request model and
> can cause bad things to happen in the scheduler. It should be possible to
> detect this at AMRM heartbeat time and throw an exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
