[
https://issues.apache.org/jira/browse/YARN-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267317#comment-16267317
]
Wangda Tan commented on YARN-7473:
----------------------------------
Thanks [~suma.shivaprasad], majorly reviewed configs and GuaranteedOrZeroPolicy
implementation.
1) observeObly can be removed from
QueueManagementDynamicEditPolicy/CapacitySchedulerConfiguration.
2) GuaranteedOrZeroCapacityOverTimePolicy
- Javadocs of the class is not accurate, please add more details to the
Javadocs.
- In Init, It's better not to cache {{parentQueueCapacities}}, instead should
always fetch from ParentQueue.getQueueCapacities. Because we don't have
guarantee that queue.queueCapacities ref won't be updated.
- ZERO_CAPACITY_ENTITLEMENT, should set capacity = 0, and maximum-capcaity
equals to leaf queue template's maximum-capacity (for all labels).
- deactivateLeafQueuesIfInActive: it's better to add a LOG.debug when
deactivate queues.
- It's better to calculate deactivatedCapacity from actual deactivated queue's
capacity for better maintenanbility. I understand this logic is correct now,
but we could have: a. queue with different capacity. b. some other actions
added before the deactive actions.
- {{isDeactivatedQueue}} is too simple to be a method.
- {{//acquires read lock on leaf queue}}, is not accurate, it just acquires
readlock of the policy itself.
- There're many variables/methods named {{*entitlement*}} inside the policy,
some of them should be renamed.
- Does the {{reinitialize}} will be used after YARN-6142? If yes, could you add
a TODO comment? And the signature should be same as init:
(CapacitySchedulerContext, ParentQueue)
- Add a TODO comment saying node label will be supported in a separate patch.
3) AutoCreatedLeafQueueConfig doesn't include maximum-am-percent, should we
include it?
> Implement Framework and policy for capacity management of auto created queues
> ------------------------------------------------------------------------------
>
> Key: YARN-7473
> URL: https://issues.apache.org/jira/browse/YARN-7473
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Attachments: YARN-7473.1.patch, YARN-7473.2.patch, YARN-7473.3.patch,
> YARN-7473.4.patch, YARN-7473.5.patch, YARN-7473.6.patch, YARN-7473.7.patch,
> YARN-7473.8.patch, YARN-7473.9.patch
>
>
> This jira mainly addresses the following
>
> 1.Support adding pluggable policies on parent queue for dynamically managing
> capacity/state for leaf queues.
> 2. Implement a default policy that manages capacity based on pending
> applications and either grants guaranteed or zero capacity to queues based on
> parent's available guaranteed capacity.
> 3. Integrate with SchedulingEditPolicy framework to trigger this periodically
> and signal scheduler to take necessary actions for capacity/queue management.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
