[
https://issues.apache.org/jira/browse/MAPREDUCE-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747790#action_12747790
]
Hemanth Yamijala commented on MAPREDUCE-893:
--------------------------------------------
As explained above, we have this feature for the properties that the framework
handles w.r.to queues - ACLs and state.
We can draw the scope of this JIRA from what exists already.
- The existing framework for refresh of ACLs and state relies on a command
*hadoop mradmin -refreshQueues*.
- This command causes QueueManager to reload the configuration from the
mapred-queues.xml file.
- If there's a syntactic or semantic error in reload, the refresh command fails
with an exception that is sent back to the hadoop mradmin command.
- Importantly, the existing configuration is untouched and the system is left
in a consistent state
- The UGI of the administrator who raised the refresh is logged to the JT log
for audit purposes.
I believe all of these generic requirements will apply for the current JIRA as
well.
The extension of scope is in the following manner:
- First the framework processes a reload of the configuration for properties it
manages.
- If it passes, the framework will call the scheduler to refresh queue
properties that are managed by the scheduler. However, it will not commit its
changes until this call succeeds.
- In MAPREDUCE-861, we suggested that schedulers which could define their
properties as key-value pairs can do so in the format we suggest
[here|https://issues.apache.org/jira/browse/MAPREDUCE-861?focusedCommentId=12744910&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12744910].
(Look at the *properties* tag under queue). The framework can pass the list of
properties per queue to the scheduler, maybe as a Map of queue-name and
properties.
- If the scheduler cannot process this information, it will throw an error (via
an exception or a return value) and discard the changes itself. The framework
will likewise discard it own changes and return error to the client with an
appropriate message.
- It is possible that some of the properties may not be refresh-able. For e.g.
we are not going to handle new queues getting added or deleted. I think we
should give a return value back to the refreshQueues to indicate this.
Does this work ?
> Provide an ability to refresh queue configuration without restart.
> ------------------------------------------------------------------
>
> Key: MAPREDUCE-893
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-893
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Hemanth Yamijala
>
> While administering a cluster using multiple queues, administrators feel a
> need to refresh queue properties on the fly without needing to restart the
> JobTracker. This is partially supported for some properties such as queue
> ACLs (HADOOP-5396) and state (HADOOP-5913). The idea is to extend the
> facility to refresh other queue properties as well, including scheduler
> properties.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
