Karthik Kambatla commented on YARN-3306:

Rephrasing what I said earlier, I am perfectly fine with staging this work in 

My only concern is with shipping 2.8.0 with this code, especially if it is not 
used by anything else (Capacity, Fair or Fifo schedulers). 
We ll be expected to maintain API and wire compatibility. We could handle API 
by not referring to it as Public-Stable, but we don't have a to do that with 
wire compat.

bq. As far as I can tell, separating this out into a branch is a net negative- 
there is overhead to doing so and it runs contrary to the iterative approach 
we're trying to take, without providing any clear benefit.
The overhead is minimal. Provides benefit. Not sure how it interferes with the 
iterative approach, especially if you can merge in functionality as and when it 
is ready (we thought enough about API and wire). If you really think branches 
are evil, at least we could work on trunk first? 

As a community, I think we should be able to cut a branch-2.8 today, fix 
remaining blockers, and create an RC. I know it is not that simple and we have 
to jump through a few hoops, but we shouldn't be blocked from doing a release 
if one person working on a feature goes on vacation. 

I would have liked us to have worked on branches for some past features, e.g. 
sharedcache - mostly isolated feature I was involved with, and many others. 

> [Umbrella] Proposing per-queue Policy driven scheduling in YARN
> ---------------------------------------------------------------
>                 Key: YARN-3306
>                 URL: https://issues.apache.org/jira/browse/YARN-3306
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: PerQueuePolicydrivenschedulinginYARN.pdf
> Scheduling layout in Apache Hadoop YARN today is very coarse grained. This 
> proposal aims at converting today’s rigid scheduling in YARN to a per­-queue 
> policy driven architecture.
> We propose the creation of a c​ommon policy framework​ and implement a​common 
> set of policies​ that administrators can pick and chose per queue
>  - Make scheduling policies configurable per queue
>  - Initially, we limit ourselves to a new type of scheduling policy that 
> determines the ordering of applications within the l​eaf ­queue
>  - In the near future, we will also pursue parent­ queue level policies and 
> potential algorithm reuse through a separate type of policies that control 
> resource limits per queue, user, application etc.

This message was sent by Atlassian JIRA

Reply via email to