[
https://issues.apache.org/jira/browse/HADOOP-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-3628:
-----------------------------------
Attachment: hadoop-3628.patch
This patch is
* updated to SVN HEAD, including the scheduling changes to JobTracker
* incorporates a fix to HADOOP-3842, the race condition in job tracker
scheduler startup
* brings the TaskTracker and MiniMRCluster into the lifecycle (though
MiniDFSCluster and MiniMR Cluster are not yet defined as Service classes
themselves)
* tunes some of the tests to shut down more elegantly
Here is the list of files in the diff
src/core/org/apache/hadoop/util/Service.java
src/hdfs/org/apache/hadoop/hdfs/server/datanode/DataNode.java
src/hdfs/org/apache/hadoop/hdfs/server/namenode/NameNode.java
src/mapred/org/apache/hadoop/mapred/JobEndNotifier.java
src/mapred/org/apache/hadoop/mapred/JobTracker.java
src/mapred/org/apache/hadoop/mapred/TaskTracker.java
src/test/org/apache/hadoop/fs/TestCopyFiles.java
src/test/org/apache/hadoop/fs/TestFileSystem.java
src/test/org/apache/hadoop/hdfs/MiniDFSCluster.java
src/test/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
src/test/org/apache/hadoop/mapred/MiniMRCluster.java
src/test/org/apache/hadoop/mapred/TestMRServerPorts.java
src/test/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
src/test/org/apache/hadoop/mapred/TestRackAwareTaskPlacement.java
src/test/org/apache/hadoop/mapred/TestTaskTrackerCleanupQueue.java
> Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-3628
> URL: https://issues.apache.org/jira/browse/HADOOP-3628
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs, mapred
> Affects Versions: 0.19.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: AbstractHadoopComponent.java, hadoop-3628.patch,
> hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch
>
>
> I'd like to propose we have a standard interface for hadoop components, the
> things that get started or stopped when you bring up a namenode. currently,
> some of these classes have a stop() or shutdown() method, with no standard
> name/interface, but no way of seeing if they are live, checking their health
> of shutting them down reliably. Indeed, there is a tendency for the spawned
> threads to not want to die; to require the entire process to be killed to
> stop the workers.
> Having a standard interface would make it easier for
> * management tools to manage the different things
> * monitoring the state of things
> * subclassing
> The latter is interesting as right now TaskTracker and JobTracker start up
> threads in their constructor; that's very dangerous as subclasses may have
> their methods called before they are full initialised. Adding this interface
> would be the right time to clean up the startup process so that subclassing
> is less risky.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
