[ 
https://issues.apache.org/jira/browse/YARN-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263089#comment-17263089
 ] 

Sunil G edited comment on YARN-10559 at 1/12/21, 5:42 AM:
----------------------------------------------------------

[~ananyo_rao] Thanks for the efforts. Few minor comments.
 # Please take a look at the newly introduced 7 checkstyle warnings and help to 
check how many of them can be fixed. 
[https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/457/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt]
 # In setFairShareForApps, there could be some chances for divide by 0. Let me 
quote those here.
{code:java}
int numOfAppsInQueue = tq.leafQueue.getAllApplications().size();
Resource fairShareAcrossApps = Resources.divideAndCeil(
  this.rc, queueReassignableResource, numOfAppsInQueue);
{code}
We are handling this internally. but still, it's better to add a check.
 # It's better to ensure two points here. We should get a non-negative resource 
and its better to check & ensurefairShareForApp cannot be 0
{code:java}
Resource fairShareForApp = Resources.min(
            rc, clusterResource, fairShareAcrossApps, fairShareWithinUL);{code}


was (Author: sunilg):
[~ananyo_rao] Thanks for the efforts. Few minor comments.
 # Please take a look at the newly introduced 7 checkstyle warnings and help to 
check how many of them can be fixed. 
[https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/457/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt]
 # In setFairShareForApps, there could be some chances for divide by 0. Let me 
quote those here.
{code:java}
int numOfAppsInQueue = tq.leafQueue.getAllApplications().size();
Resource fairShareAcrossApps = Resources.divideAndCeil(
  this.rc, queueReassignableResource, numOfAppsInQueue);
{code}
We are handling this internally. but still, it's better to add a check.

 # It's better to ensure two points here. We should get a non-negative resource 
and its better to check & ensurefairShareForApp cannot be 0
{code:java}
Resource fairShareForApp = Resources.min(
            rc, clusterResource, fairShareAcrossApps, fairShareWithinUL);{code}

> Fair sharing intra-queue preemption support in Capacity Scheduler
> -----------------------------------------------------------------
>
>                 Key: YARN-10559
>                 URL: https://issues.apache.org/jira/browse/YARN-10559
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>    Affects Versions: 3.1.4
>            Reporter: VADAGA ANANYO RAO
>            Assignee: VADAGA ANANYO RAO
>            Priority: Major
>         Attachments: FairOP_preemption-design_doc_v1.pdf, 
> FairOP_preemption-design_doc_v2.pdf, YARN-10559.0001.patch, 
> YARN-10559.0002.patch, YARN-10559.0003.patch, YARN-10559.0004.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Usecase:
> Due to the way Capacity Scheduler preemption works, If a single user submits 
> a large application to a queue (using 100% of resources), that job will not 
> be preempted by future applications from the same user within the same queue. 
> This implies that the later applications will be forced to wait for 
> completion of the long running application. This prevents multiple long 
> running, large, applications from running concurrently.
> Support fair sharing among apps while preempting applications from same queue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to