[
https://issues.apache.org/jira/browse/YARN-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13942122#comment-13942122
]
Arun C Murthy edited comment on YARN-1051 at 3/20/14 6:50 PM:
--------------------------------------------------------------
More color on why I prefer priorities for reservations rather than
adding/removing queues...
In vast majority of deployments, queues are an organizational/economic concept
(e.g. per-department queues are very common) and are queues (hierarchy, names
etc.) are quite stable and well recognized to point of being part of the
institutional memory.
If we rely on adding/removing queues to provide reservations, I'm concerned it
will cause some confusion among both admins and users. For e.g. a user/admin
trying to debug his application will be quite challenged to figure
demand/supply of resources when he has to go back in time to reconstruct a
programmatically generated queue hierarchy, particularly after it's long gone.
Priorities, OTOH, is quite a familiar concept to admins (think unix 'nice');
and more importantly is a natural fit to the problem at hand i.e. temporally
increase/decrease the priority of the application based on it's reservation at
a point in time.
Furthermore, as I said previously, priorities are an often requested feature -
especially by admins.
was (Author: acmurthy):
More color on why I prefer priorities for reservations rather than
adding/removing queues...
In vast majority of deployments, queues are an organizational/economic concept
(e.g. per-department queues are very common) and are queues (hierarchy, names
etc.) are quite stable and well recognized and part of the institutional memory.
If we rely on adding/removing queues to provide reservations, I'm concerned it
will cause some confusion among both admins and users. For e.g. a user/admin
trying to debug his application will be quite challenged to figure
demand/supply of resources when he has to go back in time to reconstruct a
programmatically generated queue hierarchy, particularly after it's long gone.
Priorities, OTOH, is quite a familiar concept to admins (think unix 'nice');
and more importantly is a natural fit to the problem at hand i.e. temporally
increase/decrease the priority of the application based on it's reservation at
a point in time.
Furthermore, as I said previously, priorities are an often requested feature -
especially by admins.
> YARN Admission Control/Planner: enhancing the resource allocation model with
> time.
> ----------------------------------------------------------------------------------
>
> Key: YARN-1051
> URL: https://issues.apache.org/jira/browse/YARN-1051
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler, resourcemanager, scheduler
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-1051-design.pdf, curino_MSR-TR-2013-108.pdf,
> techreport.pdf
>
>
> In this umbrella JIRA we propose to extend the YARN RM to handle time
> explicitly, allowing users to "reserve" capacity over time. This is an
> important step towards SLAs, long-running services, workflows, and helps for
> gang scheduling.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
