[
https://issues.apache.org/jira/browse/YARN-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472200#comment-16472200
]
Ruslan Dautkhanov commented on YARN-6567:
-----------------------------------------
My comment on YARN-5387 seems to match what this Jira is trying to address.
Thanks!
> Flexible Workload Management
> ----------------------------
>
> Key: YARN-6567
> URL: https://issues.apache.org/jira/browse/YARN-6567
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
> Reporter: Ajai Omtri
> Priority: Minor
> Labels: features
>
> Yarn workload management can be little more dynamic.
> 1. Create yarn pool by specifying more than one Secondary AD group.
> Scenario:
> In a multi-tenant cluster there can be hundreds of AD groups per tenant and
> hundreds of users per AD group. We want a way to group like workloads into
> single yarn pool by specifying multiple secondary AD Groups.
> Ex: All the ETL workloads of tenants needs to go into one yarn pool. This
> requires addition of all ETL related AD groups into one yarn pool.
> 2. Demotions
> Scenario: A particular workload/job has been started in a high priority yarn
> pool based on the assumption that it would finish quickly but due to some
> data issue/change in the code/query etc. - now it is running longer and
> consuming high amounts of resources for long time. In this case we want to
> demote this to a lower resource allocated yarn pool. We don’t want this one
> run-away workload/job to dominate the cluster because our assumption was
> wrong.
> Ex: If any workload in the yarn pool runs for X minutes and/or consumes Y
> resources either alert me or push to another yarn pool. User can keep
> demoting and can push to a yarn pool which has capped resources - like
> Penalty box.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
