[
https://issues.apache.org/jira/browse/YARN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219692#comment-16219692
]
Wangda Tan commented on YARN-4511:
----------------------------------
[~haibo.chen],
ContainerUpdateContext#swapContainer
1) Instead of using assert, should we throw exception? Assertions will be
removed at the runtime.
2) I'm not sure why it need acquire Node and do operations, the original
purpose of swapContainers/ContainerUpdateContext is (I believe you understand
the code, just to make sure we have no differences here):
a. When a promotion or demotion request comes (same as increase/decrease
container), ContainerUpdateContext calculate resource differences (for example,
promote a 2G opportunistic container means request a 2G node-local request)and
send request to scheduler.
b. Scheduler handles the increase/decrease request, which creates a new
Container and AM pulls it.
- b.1. If it is a increase request, RM changes internal resource accounting
including SchedulerNode/Queue/Application, etc.
- b.2. If it is a decrease request, RM will not change resource accounting
immediately. Instead, inside ContainerUpdateContext#swapContainer, it set
to-be-released resource to tempContainerToKill. Which will be sent to scheduler
to release async.
So my question is, is it possible to avoid the sync lock of SchedulerNode
inside swapContainer? Which looks dangerous and potentially make implementation
complicated.
I do want to review other part of the code (such as SchedulerNode) in one shot,
however I found they might be related to the swapContainer implementation.
Please let me know if you want a conf call to discuss this easier.
> Common scheduler changes supporting scheduler-specific implementations
> ----------------------------------------------------------------------
>
> Key: YARN-4511
> URL: https://issues.apache.org/jira/browse/YARN-4511
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Haibo Chen
> Attachments: YARN-4511-YARN-1011.00.patch,
> YARN-4511-YARN-1011.01.patch, YARN-4511-YARN-1011.02.patch,
> YARN-4511-YARN-1011.03.patch, YARN-4511-YARN-1011.04.patch,
> YARN-4511-YARN-1011.05.patch, YARN-4511-YARN-1011.06.patch,
> YARN-4511-YARN-1011.07.patch, YARN-4511-YARN-1011.08.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
