[
https://issues.apache.org/jira/browse/YARN-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057106#comment-15057106
]
Jian He edited comment on YARN-4138 at 12/15/15 1:06 AM:
---------------------------------------------------------
{code}
SchedContainerChangeRequest decreaseRequest =
new SchedContainerChangeRequest(
schedulerNode, rmContainer,
rmContainer.getLastConfirmedResource());
decreaseContainer(decreaseRequest,
getCurrentAttemptForContainer(containerId));
{code}
- this scenario may cause resource accounting wrong, correct me if I'm wrong:
1) AM asks increase 2G -> 8G
2) AM does not increase the container, and asks decrease to 1G
3) LastConfirmedResource becomes 1G
4) In the meantime, containerIncreaseExpiration logic is triggered and
rollbackContainerResource is invoked. In this case the resource Delta becomes
positive even in decrease case, but some code is assuming the decrease to be
negative, which may cause resource accounting wrong ?
{code}
// Delta capacity is negative when it's a decrease request
Resource absDelta = Resources.negate(decreaseRequest.getDeltaCapacity());
{code}
- I have a question about the API semantics for the above mentioned scenario.
According to the AMRMClient#requestContainerResourceChange API, the previous
pending resource-change-request should be cancelled. Essentially, the
semantics is a setter API. In that sense, the previous 8G should be cancelled.
With this approach, both resource-change-requests are cancelled. That is,10 min
later after the expiration is triggered, user will suddenly see its container
decreased to 2 GB. will this confuse the user ?
- revert format only changes in RMContainerChangeResourceEvent
was (Author: jianhe):
{code}
SchedContainerChangeRequest decreaseRequest =
new SchedContainerChangeRequest(
schedulerNode, rmContainer,
rmContainer.getLastConfirmedResource());
decreaseContainer(decreaseRequest,
getCurrentAttemptForContainer(containerId));
{code}
- this scenario may cause resource accounting wrong, correct me if I'm wrong:
1) AM asks increase 2G -> 8G
2) AM does not increase the container, and asks decrease to 1G
3) LastConfirmedResource becomes 1G
4) In the meantime, containerIncreaseExpiration logic is triggered and
rollbackContainerResource is invoked. In this case the resource Delta becomes
positive even in decrease case, but some code is assuming the decrease to be
negative, which may cause resource accounting wrong ?
{code}
// Delta capacity is negative when it's a decrease request
Resource absDelta = Resources.negate(decreaseRequest.getDeltaCapacity());
{code}
- I have a question about the API semantics for the above mentioned scenario.
According to the AMRMClient#requestContainerResourceChange API, the previous
pending resource-change-request should be cancelled. Essentially, the
semantics is a setter API. In that sense, the previous 8G should be cancelled.
With this approach, both resource-change-requests are cancelled. That is,10 min
later after the expiration is triggered, user will suddenly see its container
decreased to 2 GB. will this confuse the user ?
- revert RMContainerChangeResourceEvent
> Roll back container resource allocation after resource increase token expires
> -----------------------------------------------------------------------------
>
> Key: YARN-4138
> URL: https://issues.apache.org/jira/browse/YARN-4138
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, nodemanager, resourcemanager
> Reporter: MENG DING
> Assignee: MENG DING
> Attachments: YARN-4138-YARN-1197.1.patch, YARN-4138-YARN-1197.2.patch
>
>
> In YARN-1651, after container resource increase token expires, the running
> container is killed.
> This ticket will change the behavior such that when a container resource
> increase token expires, the resource allocation of the container will be
> reverted back to the value before the increase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
