[
https://issues.apache.org/jira/browse/YARN-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13942122#comment-13942122
]
Arun C Murthy edited comment on YARN-1051 at 3/20/14 6:50 PM:
--
More color on why I prefer priorities for reservations rather than
adding/removing queues...
In vast majority of deployments, queues are an organizational/economic concept
(e.g. per-department queues are very common) and are queues (hierarchy, names
etc.) are quite stable and well recognized to point of being part of the
institutional memory.
If we rely on adding/removing queues to provide reservations, I'm concerned it
will cause some confusion among both admins and users. For e.g. a user/admin
trying to debug his application will be quite challenged to figure
demand/supply of resources when he has to go back in time to reconstruct a
programmatically generated queue hierarchy, particularly after it's long gone.
Priorities, OTOH, is quite a familiar concept to admins (think unix 'nice');
and more importantly is a natural fit to the problem at hand i.e. temporally
increase/decrease the priority of the application based on it's reservation at
a point in time.
Furthermore, as I said previously, priorities are an often requested feature -
especially by admins.
was (Author: acmurthy):
More color on why I prefer priorities for reservations rather than
adding/removing queues...
In vast majority of deployments, queues are an organizational/economic concept
(e.g. per-department queues are very common) and are queues (hierarchy, names
etc.) are quite stable and well recognized and part of the institutional memory.
If we rely on adding/removing queues to provide reservations, I'm concerned it
will cause some confusion among both admins and users. For e.g. a user/admin
trying to debug his application will be quite challenged to figure
demand/supply of resources when he has to go back in time to reconstruct a
programmatically generated queue hierarchy, particularly after it's long gone.
Priorities, OTOH, is quite a familiar concept to admins (think unix 'nice');
and more importantly is a natural fit to the problem at hand i.e. temporally
increase/decrease the priority of the application based on it's reservation at
a point in time.
Furthermore, as I said previously, priorities are an often requested feature -
especially by admins.
YARN Admission Control/Planner: enhancing the resource allocation model with
time.
--
Key: YARN-1051
URL: https://issues.apache.org/jira/browse/YARN-1051
Project: Hadoop YARN
Issue Type: Improvement
Components: capacityscheduler, resourcemanager, scheduler
Reporter: Carlo Curino
Assignee: Carlo Curino
Attachments: YARN-1051-design.pdf, curino_MSR-TR-2013-108.pdf,
techreport.pdf
In this umbrella JIRA we propose to extend the YARN RM to handle time
explicitly, allowing users to reserve capacity over time. This is an
important step towards SLAs, long-running services, workflows, and helps for
gang scheduling.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
