[ https://issues.apache.org/jira/browse/YARN-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392793#comment-15392793 ]
Subru Krishnan commented on YARN-4359: -------------------------------------- Thanks [~imenache]/[~jyaniv] for working on this as it's a critical perf improvement. I apologize for the delay in reviewing. Overall the patch looks good, just a few comments: * An equality check, i.e. _allocationEndTime != stageStartTime_ in {{IterativePlanner::validateOrderNoGap}} should be more general than the present inequality ones. * The {{StageAllocatorLowCostAligned::getDurationInterval}} is quite dense. Consider simplifying it, especially the corner cases and/or adding more code comments. * Nit: You can have a separate method for computing how many gangs can fit in {{StageAllocatorLowCostAligned::getDurationInterval}} About the unit tests, we should have one for each change we are introducing in the patch: * Primary enhancement - using RLE representation instead of time interval, this should be covered by a no regression check on existing tests which is the case. * Adding the ability to allocate from left to right also; I see new tests for this. * I do not see one that covers the fix in {{IterativePlanner::validateOrderNoGap}}. * We should also add one to validate the corner cases in {{StageAllocatorLowCostAligned::getDurationInterval}} . > Update LowCost agents logic to take advantage of YARN-4358 > ---------------------------------------------------------- > > Key: YARN-4359 > URL: https://issues.apache.org/jira/browse/YARN-4359 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager > Reporter: Carlo Curino > Assignee: Ishai Menache > Attachments: YARN-4359.0.patch, YARN-4359.10.patch, > YARN-4359.3.patch, YARN-4359.4.patch, YARN-4359.5.patch, YARN-4359.6.patch, > YARN-4359.7.patch, YARN-4359.8.patch, YARN-4359.9.patch > > > Given the improvements of YARN-4358, the LowCost agent should be improved to > leverage this, and operate on RLESparseResourceAllocation (ideally leveraging > the improvements of YARN-3454 to compute avaialable resources) -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org