Craig Welch commented on YARN-3318:


SchedulerProcessEvents replaced with containerAllocated and containerReleased
Serial and SerialEpoch replaced with compareInputOrderTo(), which is the option 
2 for addressing it which we settled on offline
Added addSchedulerProcess/removeSchedulerProcess/addAllSchedulerProcesses
Changed configuration so that 
will setup the fair configuration, "fifo" will setup fifo, "fair+fifo" will 
setup compound fair + fifo, etc.  It is possible to setup a custom ordering 
policy class using a different configuration, but the base one will handle the 
"friendly" setup.

bq. It is not entirely clear how the ordering and limits work together - as one 
policy with multiple facets or multiple policy types
This should be modeled as different types of policies, so that they can each 
focus on their particular purpose and avoid a repetition of the intermingling 
which has made it difficult to mix, match, and share capabilities.  Having 
multiple policy types is essential to make it easy to combine them as needed.
bq. let's split the patch that exposes this to the client side / web UI and in 
the API records into its own JIRA...premature to support this as a publicly 
supportable configuration...
The goal is to make this available quickly but iteratively, keeping the changes 
small but making them available for use and feedback.  Clearly we can mark 
things "unstable", communicate that they are not fully mature/subject to 
change/should be used gently, but we will need to make it possible to activate 
the feature and use it in order to accomplish the use and feedback.  We should 
grow it organically, gradually, iteratively, think of it is a facet of the 
policy framework hooked up and available but with more to follow
bq. ...SchedulableEntity better... well, I'd actually talked [~leftnoteasy] 
into SchedulerProcess :-)   So, we can chew on this a bit more & see where we go
bq. You add/remove applications to/from LeafQueue's policy but addition/removal 
of containers is an event...
This has been factored differently along [~leftnoteasy]'s suggestion, it should 
now be consistent
bq. The notion of a comparator doesn't make sense to an admin. It is simply a 
Have modeled "policy configuration" differently, so "comparator" is out of 
sight (see above).  
bq.  Depending on how ordering and limits come together, they may become 
properties of a policy
I expect them to be distinct, this is specifically an "ordering-policy", limits 
will be other types of "limit-policy"(ies)

patch with these changes to follow in a few...

> Create Initial OrderingPolicy Framework, integrate with CapacityScheduler 
> LeafQueue supporting present behavior
> ---------------------------------------------------------------------------------------------------------------
>                 Key: YARN-3318
>                 URL: https://issues.apache.org/jira/browse/YARN-3318
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>         Attachments: YARN-3318.13.patch, YARN-3318.14.patch, 
> YARN-3318.17.patch, YARN-3318.34.patch, YARN-3318.35.patch, YARN-3318.36.patch
> Create the initial framework required for using OrderingPolicies with 
> SchedulerApplicaitonAttempts and integrate with the CapacityScheduler.   This 
> will include an implementation which is compatible with current FIFO behavior.

This message was sent by Atlassian JIRA

Reply via email to