[
https://issues.apache.org/jira/browse/HADOOP-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633244#action_12633244
]
Steve Loughran commented on HADOOP-4218:
----------------------------------------
both these failures are triggered by the same event; inside
launchJobAndTestCounters the jobtracker gets terminated; if this is set to shut
down the filesystem client then the RPC proxy gets closed,
public synchronized void close() throws IOException {
checkOpen();
clientRunning = false;
leasechecker.close();
// close connections to the namenode
RPC.stopProxy(rpcNamenode);
}
and there, apparently goes filesystem access to that namenode, across the
entire JVM. Which seems a bit of overkill.
> JobTracker may need to close its filesystem when being terminated
> -----------------------------------------------------------------
>
> Key: HADOOP-4218
> URL: https://issues.apache.org/jira/browse/HADOOP-4218
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Steve Loughran
> Priority: Minor
>
> This is something I've been experimenting with HADOOP-3268; I'm not sure what
> the right action is here.
> -currently, the JobTracker does not close() its filesystem when it is shut
> down. This will cause it to leak filesystem references if JobTrackers are
> started and stopped in the same process.
> -The TestMRServerPorts test explicitly closes the filesystem
> jt.fs.close();
> jt.stopTracker();
> -If you move the close() operation into the stopTracker()/terminate logic,
> the filesystem gets cleaned up, but
> TestRackAwareTaskPlacement and TestMultipleLevelCaching fail with a
> FilesystemClosed error (stack traces to follow)
> Should the JobTracker close its filesystem whenever it is terminated? If so,
> there are some tests that need to be reworked slightly to not expect the
> fileystem to be live after the jobtracker is taken down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
