[
https://issues.apache.org/jira/browse/MAPREDUCE-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12908590#action_12908590
]
Devaraj Das commented on MAPREDUCE-2060:
----------------------------------------
Dan, in the implementation of FileSystem caching, you will notice that there is
a UGI component in the Key. I am referring to {noformat}static class
Key{noformat} in the class FileSystem. So even though in the JobTracker, there
is no explicit doAs in the place where the FileSystem is obtained, it will get
implicitly owned by the JobTracker's UGI.
The closeAll method iterates through the FileSystem instances and closes only
those FileSystems whose UGI matches that of the passed UGI. The UGI equality is
a reference equality on the Subject field. The remote user's subject (that gets
created as part of the UGI when the RPC call in invoked at the server) and the
JobTracker's subject wouldn't match.... I am still fighting to understand
what's going on in your setup. Can you give more details on your setup, etc.,
and a way to reproduce this.
> IOException: Filesystem closed on submitJob
> -------------------------------------------
>
> Key: MAPREDUCE-2060
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2060
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment:
> https://svn.apache.org/repos/asf/hadoop/mapreduce/tr...@994941
> https://svn.apache.org/repos/asf/hadoop/hdfs/tr...@993542
> Reporter: Dan Adkins
>
> I get the following strange error on the jobtracker when attempting to submit
> a job:
> 10/09/09 20:31:35 INFO ipc.Server: IPC Server handler 7 on 31000, call
> submitJob(job_201009092028_0001,
> hdfs://hns4.sea1.qc:21000/tmp/hadoop-mr20/mapred/staging/dadkins/.staging/job_201009092028_0001,
> org.apache.hadoop.security.credenti...@20c87621) from 10.128.130.145:49253:
> error: java.io.IOException: Filesystem closed
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:307)
> at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1212)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:494)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1491)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:395)
> at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3078)
> at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3014)
> at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:2996)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:349)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1380)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1376)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1105)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1374)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
