[
https://issues.apache.org/jira/browse/YARN-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392793#comment-15392793
]
Subru Krishnan commented on YARN-4359:
--------------------------------------
Thanks [~imenache]/[~jyaniv] for working on this as it's a critical perf
improvement.
I apologize for the delay in reviewing. Overall the patch looks good, just a
few comments:
* An equality check, i.e. _allocationEndTime != stageStartTime_ in
{{IterativePlanner::validateOrderNoGap}} should be more general than the
present inequality ones.
* The {{StageAllocatorLowCostAligned::getDurationInterval}} is quite dense.
Consider simplifying it, especially the corner cases and/or adding more code
comments.
* Nit: You can have a separate method for computing how many gangs can fit in
{{StageAllocatorLowCostAligned::getDurationInterval}}
About the unit tests, we should have one for each change we are introducing in
the patch:
* Primary enhancement - using RLE representation instead of time interval,
this should be covered by a no regression check on existing tests which is the
case.
* Adding the ability to allocate from left to right also; I see new tests for
this.
* I do not see one that covers the fix in
{{IterativePlanner::validateOrderNoGap}}.
* We should also add one to validate the corner cases in
{{StageAllocatorLowCostAligned::getDurationInterval}} .
> Update LowCost agents logic to take advantage of YARN-4358
> ----------------------------------------------------------
>
> Key: YARN-4359
> URL: https://issues.apache.org/jira/browse/YARN-4359
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, fairscheduler, resourcemanager
> Reporter: Carlo Curino
> Assignee: Ishai Menache
> Attachments: YARN-4359.0.patch, YARN-4359.10.patch,
> YARN-4359.3.patch, YARN-4359.4.patch, YARN-4359.5.patch, YARN-4359.6.patch,
> YARN-4359.7.patch, YARN-4359.8.patch, YARN-4359.9.patch
>
>
> Given the improvements of YARN-4358, the LowCost agent should be improved to
> leverage this, and operate on RLESparseResourceAllocation (ideally leveraging
> the improvements of YARN-3454 to compute avaialable resources)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
