[
https://issues.apache.org/jira/browse/YARN-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505678#comment-14505678
]
Wangda Tan commented on YARN-3319:
----------------------------------
Some minor comments about configuration part:
1) getting config mapping should be a generic method, you can use
Configuration.iterator() to get all properties of a policy.
2) Policy will handle properties such as "size-based-weight" in its own
configure(..)
3) Do you think is it better to make property in
<queue-name>.ordering-policy.<policy-name>.<property-key>? In your example,
"sizeBasedWeight" more like a property of fair policy instead of others. I
perfer add a <policy-name>
4) "sizeBasedWeight" to "size-based-weight-enabled" (or
enable-size-based-weight) to make it consistent with other configurations of
CS. Add the enable to indicate it's a boolean instead of float.
5) Make the String="size-based-weight" to a final static variable like other
configuration keys. Same of "fifo" and "fifo". Test should use these final
static variables as well
FairOrderingPolicy:
1) Mark get/setSizeBasedWeight to VisibleForTesting?
2) getMagnitude should be private
3) LOG is not used by anybody, could be removed.
Findbugs warning?
> Implement a FairOrderingPolicy
> ------------------------------
>
> Key: YARN-3319
> URL: https://issues.apache.org/jira/browse/YARN-3319
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Craig Welch
> Assignee: Craig Welch
> Attachments: YARN-3319.13.patch, YARN-3319.14.patch,
> YARN-3319.17.patch, YARN-3319.35.patch, YARN-3319.39.patch,
> YARN-3319.45.patch, YARN-3319.47.patch, YARN-3319.53.patch,
> YARN-3319.58.patch, YARN-3319.70.patch, YARN-3319.71.patch,
> YARN-3319.72.patch, YARN-3319.73.patch
>
>
> Implement a FairOrderingPolicy which prefers to allocate to
> SchedulerProcesses with least current usage, very similar to the
> FairScheduler's FairSharePolicy.
> The Policy will offer allocations to applications in a queue in order of
> least resources used, and preempt applications in reverse order (from most
> resources used). This will include conditional support for sizeBasedWeight
> style adjustment
> Optionally, based on a conditional configuration to enable sizeBasedWeight
> (default false), an adjustment to boost larger applications (to offset the
> natural preference for smaller applications) will adjust the resource usage
> value based on demand, dividing it by the below value:
> Math.log1p(app memory demand) / Math.log(2);
> In cases where the above is indeterminate (two applications are equal after
> this comparison), behavior falls back to comparison based on the application
> id, which is generally lexically FIFO for that comparison
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
