[
https://issues.apache.org/jira/browse/YARN-5468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928880#comment-15928880
]
Konstantinos Karanasos commented on YARN-5468:
----------------------------------------------
Hi [~grey], we have a design doc but we are in the process of updating it to
reflect the latest discussions -- I will upload it within the next few days.
We have had extended discussions with [~leftnoteasy] to make sure we share the
same APIs for defining constraints.
In brief, we will share the same constraint expressions (affinity and
anti-affinity), but in a way that will allow to add more expressive constraints
later.
The user will be able to define application-wide constraints when submitting
the application through the ApplicationSubmissionContext (YARN-6346).
Moreover, given that we consider using such constraints for applications with
long-running containers (exclusively, at least in the beginning), we are
planning to place containers with constraints in a holistic fashion (looking at
multiple container requests and constraints at the same time), trading some
scheduling latency for better placement decisions (given that these containers
will run for hours/days/months, scheduling latency is not as critical). This
placement will be performed outside the Capacity/Fair Scheduler to make sure we
don't affect the scheduling latency of existing applications.
> Scheduling of long-running applications
> ---------------------------------------
>
> Key: YARN-5468
> URL: https://issues.apache.org/jira/browse/YARN-5468
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacityscheduler, fairscheduler
> Reporter: Konstantinos Karanasos
> Assignee: Konstantinos Karanasos
> Attachments: YARN-5468.prototype.patch
>
>
> This JIRA is about the scheduling of applications with long-running tasks.
> It will include adding support to the YARN for a richer set of scheduling
> constraints (such as affinity, anti-affinity, cardinality and time
> constraints), and extending the schedulers to take them into account during
> placement of containers to nodes.
> We plan to have both an online version that will accommodate such requests as
> they arrive, as well as a Long-running Application Planner that will make
> more global decisions by considering multiple applications at once.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
