Re: Filesystem closed errors
My app isn't a map/reduce job. On Nov 25, 2008, at 9:07 PM, David B. Ritch wrote: Do you have speculative execution enabled? I've seen error messages like this caused by speculative execution. David Bryan Duxbury wrote: I have an app that runs for a long time with no problems, but when I signal it to shut down, I get errors like this: java.io.IOException: Filesystem closed at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:196) at org.apache.hadoop.dfs.DFSClient.rename(DFSClient.java:502) at org.apache.hadoop.dfs.DistributedFileSystem.rename (DistributedFileSystem.java:176) The problems occur when I am trying to close open HDFS files. Any ideas why I might be seeing this? I though it was because I was abruptly shutting down without giving the streams a chance to get closed, but after some refactoring, that's not the case. -Bryan
Re: Filesystem closed errors
I'm fairly certain that I'm not closing the Filesystem anywhere. That said, the issue you pointed at could somehow be connected. On Nov 25, 2008, at 9:11 PM, Hong Tang wrote: Does your code ever call fs.close()? If so, https:// issues.apache.org/jira/browse/HADOOP-4655 might be relevant to your problem. On Nov 25, 2008, at 9:07 PM, David B. Ritch wrote: Do you have speculative execution enabled? I've seen error messages like this caused by speculative execution. David Bryan Duxbury wrote: I have an app that runs for a long time with no problems, but when I signal it to shut down, I get errors like this: java.io.IOException: Filesystem closed at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:196) at org.apache.hadoop.dfs.DFSClient.rename(DFSClient.java:502) at org.apache.hadoop.dfs.DistributedFileSystem.rename (DistributedFileSystem.java:176) The problems occur when I am trying to close open HDFS files. Any ideas why I might be seeing this? I though it was because I was abruptly shutting down without giving the streams a chance to get closed, but after some refactoring, that's not the case. -Bryan
Filesystem closed errors
I have an app that runs for a long time with no problems, but when I signal it to shut down, I get errors like this: java.io.IOException: Filesystem closed at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:196) at org.apache.hadoop.dfs.DFSClient.rename(DFSClient.java:502) at org.apache.hadoop.dfs.DistributedFileSystem.rename (DistributedFileSystem.java:176) The problems occur when I am trying to close open HDFS files. Any ideas why I might be seeing this? I though it was because I was abruptly shutting down without giving the streams a chance to get closed, but after some refactoring, that's not the case. -Bryan
Re: Filesystem closed errors
Do you have speculative execution enabled? I've seen error messages like this caused by speculative execution. David Bryan Duxbury wrote: I have an app that runs for a long time with no problems, but when I signal it to shut down, I get errors like this: java.io.IOException: Filesystem closed at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:196) at org.apache.hadoop.dfs.DFSClient.rename(DFSClient.java:502) at org.apache.hadoop.dfs.DistributedFileSystem.rename(DistributedFileSystem.java:176) The problems occur when I am trying to close open HDFS files. Any ideas why I might be seeing this? I though it was because I was abruptly shutting down without giving the streams a chance to get closed, but after some refactoring, that's not the case. -Bryan
Re: Filesystem closed errors
Does your code ever call fs.close()? If so, https://issues.apache.org/ jira/browse/HADOOP-4655 might be relevant to your problem. On Nov 25, 2008, at 9:07 PM, David B. Ritch wrote: Do you have speculative execution enabled? I've seen error messages like this caused by speculative execution. David Bryan Duxbury wrote: I have an app that runs for a long time with no problems, but when I signal it to shut down, I get errors like this: java.io.IOException: Filesystem closed at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:196) at org.apache.hadoop.dfs.DFSClient.rename(DFSClient.java:502) at org.apache.hadoop.dfs.DistributedFileSystem.rename (DistributedFileSystem.java:176) The problems occur when I am trying to close open HDFS files. Any ideas why I might be seeing this? I though it was because I was abruptly shutting down without giving the streams a chance to get closed, but after some refactoring, that's not the case. -Bryan
