[
https://issues.apache.org/jira/browse/YARN-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261486#comment-16261486
]
Wangda Tan commented on YARN-7473:
----------------------------------
Thanks [~suma.shivaprasad],
Comments:
1) AutoCreatedLeafQueueTemplate, better to make it to be an individual public
class.
2) Since QueueAction.ADD_QUEUE/REMOVE_QUEUE not being used, suggest to remove
all related definitions/logics, and add a note to QueueAction.
3) getInitialLeafQueueTemplate / AutoCreatedLeafQueueTemplate, etc. I found it
might be better to call it: {{AutoCreatedLeafQueueConfig}} since it is not only
a template.
4) CapacitySchedulerConfiguration:
- AUTO_CREATED_QUEUE_ENTITLEMENT_POLICY -> AUTO_CREATED_QUEUE_MANAGEMENT_POLICY
- And it should share the same prefix of AUTO_CREATE_CHILD_QUEUE_ENABLED, like
auto-create-child-queue.management-policy
- DEFAULT_AUTO_CREATED_QUEUE_ENTITLEMENT_POLICY: it's better to move it to Java
package to
{{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.queuemanagement.}}
- And move configuration prefix to
yarn.scheduler.capacity.auto-create-child-queue.management-policy, such as
{{yarn.scheduler.capacity.auto-create-child-queue.management-policy.monitoring_interval}}
5) CSQueueUtils#capacitiesSanityCheck, could be private.
6) QueueManagementDynamicEditPolicy#observeOnly, I'm not sure if it is useful.
From my experience of preemption, I don't see people are using this. I suggest
to remove it.
Pending review: GuaranteedOrZeroCapacityOverTimePolicy.
> Implement Framework and policy for capacity management of auto created queues
> ------------------------------------------------------------------------------
>
> Key: YARN-7473
> URL: https://issues.apache.org/jira/browse/YARN-7473
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Attachments: YARN-7473.1.patch, YARN-7473.2.patch, YARN-7473.3.patch,
> YARN-7473.4.patch, YARN-7473.5.patch
>
>
> This jira mainly addresses the following
>
> 1.Support adding pluggable policies on parent queue for dynamically managing
> capacity/state for leaf queues.
> 2. Implement a default policy that manages capacity based on pending
> applications and either grants guaranteed or zero capacity to queues based on
> parent's available guaranteed capacity.
> 3. Integrate with SchedulingEditPolicy framework to trigger this periodically
> and signal scheduler to take necessary actions for capacity/queue management.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
