[jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
[ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524290#comment-14524290 ] Xuan Gong commented on YARN-1990: - Close this ticket based on [~curino]'s comment. > Track time-to-allocation for different size containers > --- > > Key: YARN-1990 > URL: https://issues.apache.org/jira/browse/YARN-1990 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Carlo Curino >Assignee: Carlo Curino > > Allocation of Large Containers are notoriously problematic, as smaller > containers can more easily grab resources. > The proposal for this JIRA is to maintain a map of container sizes, and > time-to-allocation, that can be used as: > * general insight on cluster behavior, > * to inform the reservation-system, and allows us to "account for delays" in > allocation, so that the user reservation is respected regardless the size of > containers requested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
[ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166099#comment-14166099 ] Carlo Curino commented on YARN-1990: YARN-1051 is committed. If someone tries it and finds issues with the time-to-allocation, we can re-hash this. Otherwise we can just close this in a while. > Track time-to-allocation for different size containers > --- > > Key: YARN-1990 > URL: https://issues.apache.org/jira/browse/YARN-1990 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Carlo Curino >Assignee: Carlo Curino > > Allocation of Large Containers are notoriously problematic, as smaller > containers can more easily grab resources. > The proposal for this JIRA is to maintain a map of container sizes, and > time-to-allocation, that can be used as: > * general insight on cluster behavior, > * to inform the reservation-system, and allows us to "account for delays" in > allocation, so that the user reservation is respected regardless the size of > containers requested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
[ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111535#comment-14111535 ] Carlo Curino commented on YARN-1990: We had a simple implementation for this, using QueueMetrics and maintaining a map of delays (using SampleQuantile), tracking start-end of the wait-time by catching reserve() and unreserve() calls. In our test environment, it didn't seem to matter much. We might further investigate, and post patches after YARN-1051 is committed. > Track time-to-allocation for different size containers > --- > > Key: YARN-1990 > URL: https://issues.apache.org/jira/browse/YARN-1990 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Carlo Curino >Assignee: Carlo Curino > > Allocation of Large Containers are notoriously problematic, as smaller > containers can more easily grab resources. > The proposal for this JIRA is to maintain a map of container sizes, and > time-to-allocation, that can be used as: > * general insight on cluster behavior, > * to inform the reservation-system, and allows us to "account for delays" in > allocation, so that the user reservation is respected regardless the size of > containers requested. -- This message was sent by Atlassian JIRA (v6.2#6252)
