[
https://issues.apache.org/jira/browse/YARN-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486312#comment-14486312
]
Craig Welch commented on YARN-3318:
-----------------------------------
bq. 1) Regarding OrderingPolicy and SchedulingOrder responsibilities:
Scheduling order has multiple purposes, including:
1. Housing supporting code for using policies common across schedulers, e.g. a
common implementation of behavior
2. Allowing for the composition of multiple policies together to accomplish
desired queue behavior - it is awkward to factor the functionality in
SchedulingOrder down into the policies, as multiple policies are in play for
one instance of the logic in SchedulingOrder
Although I mentioned that it could be made abstract some day if needed, that's
not it's current purpose, the above are.
bq. ...Looking at methods of OrderingPolicy, most of them are just pass through
parameters to OrderingPolicy, and rest of them are instantiation
OrderingPolicies...
Well, no, it has quite a lot of implementation logic around managing the
SchedulerProcess collection and the interactions between it and multiple
policies, it is certainly not limited to Factory operations
bq. OrderingPolicy should be a per-queue instance or global library
OrderingPolicies are per-queue and stateful in terms of configuration specific
to that queue configuration. For the reasons mentioned above regarding the
composition of policies, they do not (and should not) maintain queue state
(scheduler processes, etc).
bq. Suggestion about OrderingPolicy interface design (if you agree with 1/2):
I don't agree, so skipping the section. The essential thing that I think is
being missed here is that there is an intentional desire to compose ordering
policies for a queue to achieve behavior - so priority + fifo, or fair + fifo,
etc, and for that reason it is not appropriate to place the management of the
collection of processes shared amongst policies into the policy implementation
- it belongs outside, as it is today, in SchedulingOrder. Mixing these
together defeats composition and also mixes concerns, making the code more (not
less) complex and certainly less clean in terms of separation of concern and
overall design and flow.
bq. ...CompoundOrderingPolicy is implemenation detail for FairOrderingPolicy,
don't need put in the patch...
Not is isn't, it's a feature of the generalized framework to support multiple
policies being composed for a queue, it's not specific to fairness at all
(fairness may be the first user, but so might priority - in any case, any set
of policies may use it, it's not specific to any one of them, and therefore is
framework...)
bq. ...About spliting SchedulableProcess to App and Queue...
I stand by my earlier explanation (and don't see anything here which alters
it...), I anticipate that with the current factoring of SchedulerProcess we
won't have to subtype it to support Queues. That said, the right time to do
that is when we are adding such support, "anticipatory complexity" is the worst
kind. It is factored such that adding the subtyping should be additive if it
needs to happen, so there is no need to anticipate it now (the room is there to
add it, which is all we need. We should wait to add it until we know we need
it).
bq. ...As I mentioned before, use ResourceUsage is much better...
As I mentioned before, it doesn't presently supply the needed functionality,
when it does we can convert to it.
> Create Initial OrderingPolicy Framework and FifoOrderingPolicy
> --------------------------------------------------------------
>
> Key: YARN-3318
> URL: https://issues.apache.org/jira/browse/YARN-3318
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Craig Welch
> Assignee: Craig Welch
> Attachments: YARN-3318.13.patch, YARN-3318.14.patch,
> YARN-3318.17.patch, YARN-3318.34.patch, YARN-3318.35.patch,
> YARN-3318.36.patch, YARN-3318.39.patch, YARN-3318.45.patch,
> YARN-3318.47.patch, YARN-3318.48.patch
>
>
> Create the initial framework required for using OrderingPolicies and an
> initial FifoOrderingPolicy
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
