[
https://issues.apache.org/jira/browse/SLIDER-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980713#comment-14980713
]
Steve Loughran commented on SLIDER-955:
---------------------------------------
well done for finding this problem!
I'll have to look @ the slider code here, as we really try to track outstanding
requests by container priority; I guess the issue is that when we try to cancel
a request, the requirements of the request to cancel don't match the
(multiplied) view.
I think we go for fix #2 here, multiply up. With fix #1 things will fail
depending on the cluster size, for no obvious reason. And as you node -yarn is
multiplying up automatically anyway.
What slider would do just premultiply the value from resources.json when it
loads in the resources
> fail to track the outstandingRequest when submit an application that
> yarn.memory is not a multiple of minimum-allocation-mb
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: SLIDER-955
> URL: https://issues.apache.org/jira/browse/SLIDER-955
> Project: Slider
> Issue Type: Improvement
> Components: appmaster
> Affects Versions: Slider 0.80
> Reporter: kyungwan nam
>
> "yarn.scheduler.minimum-allocation-mb” is set to 256 in my cluster.
> I submit an application that “yarn.memory” is set to 7169 (not a multiple of
> 256)
> the resource of allocated container will be 7424(256*29) memory because the
> resource requirement is normalized to a multiple of
> "yarn.scheduler.minimum-allocation-mb” by RM.
> the resource requirement of outstandingRequest that is tracked by Slider AM
> does not match the resource of the allocated container.
> therefore, Slider AM can’t clean up the outstandingRequest and requests in
> AMRMClient.
> I think it should fix as following 1 or 2
> 1. an application should be able to submit only if “yarn.memory” is a
> multiple of "yarn.scheduler.minimum-allocation-mb”
> 2. the resource requirement of outstandingRequest is normalized to a multiple
> of "yarn.scheduler.minimum-allocation-mb”
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
