Hi,
I have 2 threads that copy file from hdfs and delete the directory
after copying the file.
In both the threads I use "FileSystem hdfs = FileSystem.get(conf);"
Once i finish copying and deleting I close the filesystem(
hdfs.close() in the finally block)
If one of threads does a FileSystem.close()(while the other thread is
still copying) the other threads stops copying and throws an error
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226)
at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:67)
at
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close(DFSClient.java:1678)
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:58)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:85)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:209)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1216)
at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1197)
Should I NOT do FileSystem.close() in the finally block ?? How I solve
this issue ?
Cheers,
Karthik
