[
https://issues.apache.org/jira/browse/YARN-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Craig Welch updated YARN-3319:
------------------------------
Description:
Implement a FairOrderingPolicy which prefers to allocate to SchedulerProcesses
with least current usage, very similar to the FairScheduler's FairSharePolicy.
The Policy will offer allocations to applications in a queue in order of least
resources used, and preempt applications in reverse order (from most resources
used). This will include conditional support for sizeBasedWeight style
adjustment
Optionally, based on a conditional configuration to enable sizeBasedWeight
(default false), an adjustment to boost larger applications (to offset the
natural preference for smaller applications) will adjust the resource usage
value based on demand, dividing it by the below value:
Math.log1p(app memory demand) / Math.log(2);
In cases where the above is indeterminate (two applications are equal after
this comparison), behavior falls back to comparison based on the application
id, which is generally lexically FIFO for that comparison
was:
Implement a Fair Comparator for the Scheduler Comparator Ordering Policy which
prefers to allocate to SchedulerProcesses with least current usage, very
similar to the FairScheduler's FairSharePolicy.
The Policy will offer allocations to applications in a queue in order of least
resources used, and preempt applications in reverse order (from most resources
used). This will include conditional support for sizeBasedWeight style
adjustment
An implementation of a Scheduler Comparator for use with the Scheduler
Comparator Ordering Policy will be built with the below comparison for ordering
applications for container assignment (ascending) and for preemption
(descending)
Current resource usage - less usage is lesser
Submission time - earlier is lesser
Optionally, based on a conditional configuration to enable sizeBasedWeight
(default false), an adjustment to boost larger applications (to offset the
natural preference for smaller applications) will adjust the resource usage
value based on demand, dividing it by the below value:
Math.log1p(app memory demand) / Math.log(2);
In cases where the above is indeterminate (two applications are equal after
this comparison), behavior falls back to comparison based on the application
name, which is lexically FIFO for that comparison (first submitted is lesser)
> Implement a FairOrderingPolicy
> ------------------------------
>
> Key: YARN-3319
> URL: https://issues.apache.org/jira/browse/YARN-3319
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Craig Welch
> Assignee: Craig Welch
> Attachments: YARN-3319.13.patch, YARN-3319.14.patch,
> YARN-3319.17.patch, YARN-3319.35.patch, YARN-3319.39.patch
>
>
> Implement a FairOrderingPolicy which prefers to allocate to
> SchedulerProcesses with least current usage, very similar to the
> FairScheduler's FairSharePolicy.
> The Policy will offer allocations to applications in a queue in order of
> least resources used, and preempt applications in reverse order (from most
> resources used). This will include conditional support for sizeBasedWeight
> style adjustment
> Optionally, based on a conditional configuration to enable sizeBasedWeight
> (default false), an adjustment to boost larger applications (to offset the
> natural preference for smaller applications) will adjust the resource usage
> value based on demand, dividing it by the below value:
> Math.log1p(app memory demand) / Math.log(2);
> In cases where the above is indeterminate (two applications are equal after
> this comparison), behavior falls back to comparison based on the application
> id, which is generally lexically FIFO for that comparison
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
