[
https://issues.apache.org/jira/browse/YARN-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901081#comment-13901081
]
Junping Du commented on YARN-1506:
----------------------------------
Agree with Arun that this is not a blocker.
Hi [~bikassaha], Thanks for you review and comments. Sorry for replying late as
just come back from a long vacation. Please see my reply below:
bq. ADMIN_RESOURCE_UPDATE instead of RESOURCE_UPDATE for the enum would help
clarify that its a forced admin update.
Ok. Will update it.
bq. Why not update the total capability here also (like we do for non-running
node). When the node reports back as healthy then we would probably need the
new resource value, right?
For node that unusable (unhealthy, LOST or decommissioned), I think it may be
simpler to just log and warn rather than do any valid change. Or user may get
confused that the node is still usable. Thoughts?
bq. Why are we doing this indirect subtraction via delta instead of simply
clusterResource-=old; clusterResource+=new. Its the same number of operations
and less confusing to read.
Good point. Will update it.
bq. I think its crucial to have a more complete test (maybe using mockRM) that
verifies the flow from admin service to the scheduler. Most interesting would
be the case when the node is full allocated and then an update reduces the
capacity. Thus resulting in -ve value of available resource on the node. I am
wary that this case may have bugs in handling the -ve value in existing
scheduler code because its unexpected. Its fine for the test to use the default
scheduler.
Agree. Although I am pretty sure it works fine so far from my offline
integration test, we have to add unit test to cover resource over-commitment
case so any changes in future won't break these assumptions.
Will update patch soon.
> Replace set resource change on RMNode/SchedulerNode directly with event
> notification.
> -------------------------------------------------------------------------------------
>
> Key: YARN-1506
> URL: https://issues.apache.org/jira/browse/YARN-1506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, scheduler
> Reporter: Junping Du
> Assignee: Junping Du
> Priority: Blocker
> Attachments: YARN-1506-v1.patch, YARN-1506-v2.patch,
> YARN-1506-v3.patch, YARN-1506-v4.patch, YARN-1506-v5.patch, YARN-1506-v6.patch
>
>
> According to Vinod's comments on YARN-312
> (https://issues.apache.org/jira/browse/YARN-312?focusedCommentId=13846087&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13846087),
> we should replace RMNode.setResourceOption() with some resource change event.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
