[
https://issues.apache.org/jira/browse/YARN-4692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154548#comment-15154548
]
Konstantinos Karanasos commented on YARN-4692:
----------------------------------------------
Thanks [~vinodkv] for the document -- I think it is a great starting point.
As we discussed, we will provide you with some additional use cases that will
help iron out some more details.
Some thoughts regarding the scheduling of containers...
I do like the idea of [~leftnoteasy] to combine application priorities and
expected task duration (YARN-1039) in order to handle different types of jobs.
That is, expected task duration is used to determine whether a task is going to
be long-running (such as a service), while application priorities are used to
decide how tasks get executed in the NM.
I think this also nicely captures the preemptability issue that was raised by
[~asuresh] without the need for extra knowledge/fields.
That said, I still have a few concerns:
* Who determines priorities? On a cluster shared by multiple users ("competing"
with each other for resources), most users will want to use the highest
priority (unless there are pricing or other incentives).
* In order to do a proper placement of services, I think there is still some
information that is missing regarding the resource needs of the service. For
instance, a service might be latency-critical. If we could capture such
information in the resource requests, we could automatically reason about
affinity and anti-affinity constraints (e.g., avoid placing two services that
are both latency critical on the same node).
Thoughts?
> [Umbrella] Simplified and first-class support for services in YARN
> ------------------------------------------------------------------
>
> Key: YARN-4692
> URL: https://issues.apache.org/jira/browse/YARN-4692
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments:
> YARN-First-Class-And-Simplified-Support-For-Services-v0.pdf
>
>
> YARN-896 focused on getting the ball rolling on the support for services
> (long running applications) on YARN.
> I’d like propose the next stage of this effort: _Simplified and first-class
> support for services in YARN_.
> The chief rationale for filing a separate new JIRA is threefold:
> - Do a fresh survey of all the things that are already implemented in the
> project
> - Weave a comprehensive story around what we further need and attempt to
> rally the community around a concrete end-goal, and
> - Additionally focus on functionality that YARN-896 and friends left for
> higher layers to take care of and see how much of that is better integrated
> into the YARN platform itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
