[jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
[ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524290#comment-14524290 ] Xuan Gong commented on YARN-1990: - Close this ticket based on [~curino]'s comment. Track time-to-allocation for different size containers --- Key: YARN-1990 URL: https://issues.apache.org/jira/browse/YARN-1990 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Carlo Curino Allocation of Large Containers are notoriously problematic, as smaller containers can more easily grab resources. The proposal for this JIRA is to maintain a map of container sizes, and time-to-allocation, that can be used as: * general insight on cluster behavior, * to inform the reservation-system, and allows us to account for delays in allocation, so that the user reservation is respected regardless the size of containers requested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
[ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166099#comment-14166099 ] Carlo Curino commented on YARN-1990: YARN-1051 is committed. If someone tries it and finds issues with the time-to-allocation, we can re-hash this. Otherwise we can just close this in a while. Track time-to-allocation for different size containers --- Key: YARN-1990 URL: https://issues.apache.org/jira/browse/YARN-1990 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Carlo Curino Allocation of Large Containers are notoriously problematic, as smaller containers can more easily grab resources. The proposal for this JIRA is to maintain a map of container sizes, and time-to-allocation, that can be used as: * general insight on cluster behavior, * to inform the reservation-system, and allows us to account for delays in allocation, so that the user reservation is respected regardless the size of containers requested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
[ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111535#comment-14111535 ] Carlo Curino commented on YARN-1990: We had a simple implementation for this, using QueueMetrics and maintaining a map of delays (using SampleQuantile), tracking start-end of the wait-time by catching reserve() and unreserve() calls. In our test environment, it didn't seem to matter much. We might further investigate, and post patches after YARN-1051 is committed. Track time-to-allocation for different size containers --- Key: YARN-1990 URL: https://issues.apache.org/jira/browse/YARN-1990 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Carlo Curino Allocation of Large Containers are notoriously problematic, as smaller containers can more easily grab resources. The proposal for this JIRA is to maintain a map of container sizes, and time-to-allocation, that can be used as: * general insight on cluster behavior, * to inform the reservation-system, and allows us to account for delays in allocation, so that the user reservation is respected regardless the size of containers requested. -- This message was sent by Atlassian JIRA (v6.2#6252)
