[
https://issues.apache.org/jira/browse/HADOOP-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713413#action_12713413
]
rahul k singh commented on HADOOP-5913:
---------------------------------------
Summary:
-This feature would provide functionality to stop and start queues in Hadoop
at runtime. All the running jobs in the queue will be
completed , and any further new jobs will not be accepted. This state would
be persisted in the configuration file.
Requirements:
- Administrators should be able to stop(stop accepting jobs) and start(start
accepting jobs)
queues at runtime.
- if a queue is stopped at runtime , it should complete all the existing
running job and stop
accepting any new jobs.
- Administators should be able to change the queue configuration to start and
stop.
- Once configuration is updated , administrators run " hadoop refreshQueues"
command to refresh the existing
queues.This configuration would be persisted across the Jobtracker restart.
- Moving all the queue info(ACLS and state(stop or running)) in a common xml
file , this would reduce the no of
commands required to change any queue settings , we can simply change this
xml file and execute "hadoop refreshQueues"
to update the queue information.
- In order to do the above , we would be renaming mapred-queue-acls.xml file
to mapred-queues.xml.
Design
-Assumptions:
- Administrators would have permission to execute the "hadoop refreshQueues"
command.
- Only settings which are mentioned in mapred-queue.xml would be changed ,
any queue settings which are not part of
mapred-queue.xml(For ex. settings in scheduler's configuration) will not be
effected.
-Summary
- Rename the"mapred-queue-acls.xml" to "mapred-queues.xml"
- Move all the queue related data from mapred-site.xml to mapred-queues.xml
Following tags would be moved.
- mapred.queue.names
- mapred.acls.enabled
- mapred.queue.<queueName>.acl-submit-job //This setting is for each queue
- mapred.queue.<queueName>.acl-administer-job //This setting is for each
queue
- Refactor the existing code to encapsulate this new mapred-queues.xml file
- Introduce the new command "hadoop refreshQueues" which reads from the
existing mapred-queues.xml
- Introduce new api in QueuesManager.java to check for state of the Queue.
- Introduction of new property for Queue in mapred-queues.xml.
"mapred.queue.<queueName>.state" values being "stopped" | "running"
> Allow administrators to be able to start and stop queues
> ---------------------------------------------------------
>
> Key: HADOOP-5913
> URL: https://issues.apache.org/jira/browse/HADOOP-5913
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Reporter: rahul k singh
>
> This feature would provide functionality to stop and start queues in Hadoop
> at runtime.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
