[
https://issues.apache.org/jira/browse/HADOOP-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632647#action_12632647
]
Steve Loughran commented on HADOOP-4218:
----------------------------------------
stack traces of tests that fail once the JobTracker closes its filesystem when
terminated
TestMultipleLevelCaching testMultiLevelCaching Error Filesystem
closed
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:197)
at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:537)
at
org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:201)
at
org.apache.hadoop.mapred.TestMultipleLevelCaching.testCachingAtLevel(TestMultipleLevelCaching.java:116)
at
org.apache.hadoop.mapred.TestMultipleLevelCaching.testMultiLevelCaching(TestMultipleLevelCaching.java:69)
TestRackAwareTaskPlacement testTaskPlacement Error Filesystem
closed
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:197)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:574)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:400)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:651)
at
org.apache.hadoop.mapred.TestRackAwareTaskPlacement.launchJobAndTestCounters(TestRackAwareTaskPlacement.java:78)
at
org.apache.hadoop.mapred.TestRackAwareTaskPlacement.testTaskPlacement(TestRackAwareTaskPlacement.java:156)
> JobTracker may need to close its filesystem when being terminated
> -----------------------------------------------------------------
>
> Key: HADOOP-4218
> URL: https://issues.apache.org/jira/browse/HADOOP-4218
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Steve Loughran
> Priority: Minor
>
> This is something I've been experimenting with HADOOP-3268; I'm not sure what
> the right action is here.
> -currently, the JobTracker does not close() its filesystem when it is shut
> down. This will cause it to leak filesystem references if JobTrackers are
> started and stopped in the same process.
> -The TestMRServerPorts test explicitly closes the filesystem
> jt.fs.close();
> jt.stopTracker();
> -If you move the close() operation into the stopTracker()/terminate logic,
> the filesystem gets cleaned up, but
> TestRackAwareTaskPlacement and TestMultipleLevelCaching fail with a
> FilesystemClosed error (stack traces to follow)
> Should the JobTracker close its filesystem whenever it is terminated? If so,
> there are some tests that need to be reworked slightly to not expect the
> fileystem to be live after the jobtracker is taken down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
