Craig Welch commented on YARN-3319:

bq. Some minor comments about configuration part

by index:
1) done
2) done
3) done - see below
bq. Do you think is it better to make property in 
Now that there is not proper composition only one policy can be active at a 
time and it shouldn't be necessary to namespace config items this way.  At the 
same time, I could see us getting back to proper composition at some point, 
where this would be helpful. I've implemented it as a prefix convention in the 
policy instead of constraining the contents of the map in the capacity 
scheduler configuration.  This is because we still support passing a class name 
as the policy type, which would make the configurations for "class name based" 
items unwieldy.  It would also allow us to have shared configuration items 
between policies if we do end up with proper composition again.  The end result 
of the configuration was as you suggested
4) done
5) done

bq. FairOrderingPolicy:
all 3 done

bq. Findbugs warning?
Failed to stage change, so it didn't make it into patch, should be there now.

> Implement a FairOrderingPolicy
> ------------------------------
>                 Key: YARN-3319
>                 URL: https://issues.apache.org/jira/browse/YARN-3319
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>         Attachments: YARN-3319.13.patch, YARN-3319.14.patch, 
> YARN-3319.17.patch, YARN-3319.35.patch, YARN-3319.39.patch, 
> YARN-3319.45.patch, YARN-3319.47.patch, YARN-3319.53.patch, 
> YARN-3319.58.patch, YARN-3319.70.patch, YARN-3319.71.patch, 
> YARN-3319.72.patch, YARN-3319.73.patch
> Implement a FairOrderingPolicy which prefers to allocate to 
> SchedulerProcesses with least current usage, very similar to the 
> FairScheduler's FairSharePolicy.  
> The Policy will offer allocations to applications in a queue in order of 
> least resources used, and preempt applications in reverse order (from most 
> resources used). This will include conditional support for sizeBasedWeight 
> style adjustment
> Optionally, based on a conditional configuration to enable sizeBasedWeight 
> (default false), an adjustment to boost larger applications (to offset the 
> natural preference for smaller applications) will adjust the resource usage 
> value based on demand, dividing it by the below value:
> Math.log1p(app memory demand) / Math.log(2);
> In cases where the above is indeterminate (two applications are equal after 
> this comparison), behavior falls back to comparison based on the application 
> id, which is generally lexically FIFO for that comparison

This message was sent by Atlassian JIRA

Reply via email to