[
https://issues.apache.org/jira/browse/YARN-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290233#comment-16290233
]
Wangda Tan commented on YARN-7457:
----------------------------------
Reassigned to [~Tao Yang],
Thanks Tao for sharing this design doc. Comments regarding the design doc:
1) Overall I think the separate request to update delay scheduling is overkill.
We should have a way to tell YARN how to choose delay scheduling policy as well
as delay parameters. Instead of changing protocol, I suggest initially we can
put it under ApplicationSubmissionContext#getAMContainerSpec#environments. With
this, we don’t have to change protocols and all existing YARN apps can use it
without editing code.
2) I’m doubt if it is a valid case that a single YARN app wants to choose
different delay policies for different SchedulerRequestKey. If it is a
non-requirement, #1 should be enough in the short term until we figure out
solid requirements.
3) The original purpose of the JIRA is to make following logics separate from
main scheduler implementation:
* AppSchedulingInfo#canDelayTo
* AppPlacementAllocator#canDelayTo
* And all delay-scheduling related logics (such as
RegularContainerAllocator#canAssign)
To clarify, this doesn’t necessarily mean we should move these logics to a
separate service. In my opinion, applications should determine how to do delay
scheduling by themselves. So to me a proper place to handle this logic is
inside AppPlacementAllocator, but feel free to suggest if you have any other
thoughts.
> Delay scheduling should be an individual policy instead of part of scheduler
> implementation
> -------------------------------------------------------------------------------------------
>
> Key: YARN-7457
> URL: https://issues.apache.org/jira/browse/YARN-7457
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Tao Yang
>
> Currently, different schedulers have slightly different delay scheduling
> implementations. Ideally we should make delay scheduling independent from
> scheduler implementation. Benefits of doing this:
> 1) Applications can choose which delay scheduling policy to use, it could be
> time-based / missed-opportunistic-based or whatever new delay scheduling
> policy supported by the cluster. Now it is global config of scheduler.
> 2) Make scheduler implementations simpler and reusable.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
