failure closing block of file
-----------------------------
Key: HADOOP-643
URL: http://issues.apache.org/jira/browse/HADOOP-643
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.7.2
Reporter: Johan Oskarson
Priority: Critical
I've been getting "failure closing block of file" on random files.
Both datanode and tasktracker running on node7. No problems with pinging.
Guess it got stuck after the NPE in DataNode.
Job cannot start because of:
java.io.IOException: failure closing block of file
/home/hadoop/mapred/system/submit_99u9cd/.job.jar.crc to node node7:50010
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.internalClose(DFSClient.java:1199)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1163)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1241)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at
org.apache.hadoop.fs.FSDataOutputStream$Summer.close(FSDataOutputStream.java:96)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at org.apache.hadoop.fs.FileUtil.copyContent(FileUtil.java:205)
at org.apache.hadoop.fs.FileUtil.copyContent(FileUtil.java:190)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:77)
at
org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:186)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:289)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:314)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:248)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:280)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at java.io.DataInputStream.readFully(DataInputStream.java:176)
at java.io.DataInputStream.readLong(DataInputStream.java:380)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.internalClose(DFSClient.java:1193)
Cxception in datanode.out on node7:
Exception in thread "[EMAIL PROTECTED]" java.lang.NullPointerException
at
org.apache.hadoop.dfs.FSDataset$FSDir.checkDirTree(FSDataset.java:162)
at
org.apache.hadoop.dfs.FSDataset$FSDir.checkDirTree(FSDataset.java:162)
at
org.apache.hadoop.dfs.FSDataset$FSVolume.checkDirs(FSDataset.java:238)
at
org.apache.hadoop.dfs.FSDataset$FSVolumeSet.checkDirs(FSDataset.java:326)
at org.apache.hadoop.dfs.FSDataset.checkDataDir(FSDataset.java:522)
at
org.apache.hadoop.dfs.DataNode$DataXceiveServer.run(DataNode.java:480)
at java.lang.Thread.run(Thread.java:595)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
