[
https://issues.apache.org/jira/browse/YARN-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408657#comment-15408657
]
Konstantinos Karanasos edited comment on YARN-4902 at 8/4/16 11:57 PM:
-----------------------------------------------------------------------
I am uploading a design document that describes our vision for scheduling
long-running applications (LRA).
It is a very initial version, but I am sharing it, so that it helps drive the
discussion.
There are overlapping bits with this JIRA (after all, up to a point, it targets
the same problem), but there are clearly new points, especially when it comes
to LRA planning.
As I had explained to [~leftnoteasy] offline during the Hadoop Summit, our
focus is not on the scheduling given affinity/anti-affinity constraints, but on
the LRA *planning*.
We did a first implementation of affinity, anti-affinity and *cardinality*
constraints, because it was required for us to proceed with the LRA planning
and nothing was available at that time.
[That said, we have already added support for cardinality and I think we have a
different support for tags (but I need to take a closer look on YARN-1042) --
let's continue the discussion at that JIRA.]
Given that Wangda marked YARN-5468 as duplicate, do you believe that the LRA
planing belongs to this or another existing JIRA?
As far as I can tell, it does not.
Let me know what you think, so that we can use the proper JIRAs and avoid
duplicate effort going forward.
Thanks.
was (Author: kkaranasos):
I am uploading a design document that describes our vision for scheduling
long-running applications (LRA).
It is a very initial version, but I am sharing it, so that it helps drive the
discussion.
There are overlapping bits with this JIRA (after all, up to a point, it targets
the same problem), but there are clearly new points, especially when it comes
to LRA planning.
As I had explained to [~leftnoteasy] offline during the Hadoop Summit, our
focus is not on the scheduling given affinity/anti-affinity constraints, but on
the LRA *planning*.
We did a first implementation of affinity, anti-affinity and *cardinality*
constraints, because it was required for us to proceed with the LRA planning
and nothing was available at that time.
[That said, we have already added support for cardinality and I think we have a
different support for tags (but I need to take a closer look on YARN-1042) --
let's continue the discussion at that JIRA.]
Given that Wangda marked YARN-5048 as duplicate, do you believe that the LRA
planing belongs to this or another existing JIRA?
As far as I can tell, it does not.
Let me know what you think, so that we can use the proper JIRAs and avoid
duplicate effort going forward.
Thanks.
> [Umbrella] Generalized and unified scheduling-strategies in YARN
> ----------------------------------------------------------------
>
> Key: YARN-4902
> URL: https://issues.apache.org/jira/browse/YARN-4902
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Wangda Tan
> Attachments: Generalized and unified scheduling-strategies in YARN
> -v0.pdf, LRA-scheduling-design.v0.pdf, YARN-5468.prototype.patch
>
>
> Apache Hadoop YARN's ResourceRequest mechanism is the core part of the YARN's
> scheduling API for applications to use. The ResourceRequest mechanism is a
> powerful API for applications (specifically ApplicationMasters) to indicate
> to YARN what size of containers are needed, and where in the cluster etc.
> However a host of new feature requirements are making the API increasingly
> more and more complex and difficult to understand by users and making it very
> complicated to implement within the code-base.
> This JIRA aims to generalize and unify all such scheduling-strategies in YARN.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
