DFSClient getting stuck intermittently giving "DFSClient: Could not complete
file filename retrying..." message
-----------------------------------------------------------------------------------------------------------------
Key: HADOOP-3830
URL: https://issues.apache.org/jira/browse/HADOOP-3830
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.18.0
Reporter: Karam Singh
While some streaming job with hadoop 0.18 and hadoop trunk, hadoop commands
getting stuck intermittently and keeps on printing message "DFSClient:
<filename>Could not complete file retrying..." in indefinite loop .
I have not faced this problem in hadoop 0.17,2.
On one occasion after copying to hdfs second command "hadoop fs - put file.txt
input/in.txt" got stuck in hadoop 0.18 and hadoop trunk version
Following is the message -:
[
08/07/24 05:03:58 INFO hdfs.DFSClient: Could not complete file /
user/userid/input/rin.txt retrying...
08/07/24 05:03:59 INFO hdfs.DFSClient: Could not complete file /
user/userid/input/rin.txt retrying...
08/07/24 05:03:59 INFO hdfs.DFSClient: Could not complete file /
user/userid/input/rin.txt retrying...
]
On second occasion command -: "hadoop jar hadoop-0.19.0-dev-examples.jar
randomtextwriter rwout" got stuck with following -:
[
08/07/24 08:39:06 INFO hdfs.DFSClient: Could not complete file
/mparedsystem/userid/mapredsystem/clusterid/ job_200807240828_0028/job.xml
retrying...
08/07/24 08:39:06 INFO hdfs.DFSClient: Could not complete file
/mparedsystem/userid/mapredsystem/clusterid/ job_200807240828_0028/job.xml
retrying...
08/07/24 08:39:06 INFO hdfs.DFSClient: Could not complete file
/mparedsystem/userid/mapredsystem/clusterid/ job_200807240828_0028/job.xml
retrying...
08/07/24 08:39:07 INFO hdfs.DFSClient: Could not complete file
/mparedsystem/userid/mapredsystem/clusterid/ job_200807240828_0028/job.xml
retrying...
]
This is intermittent and there is no specific pattern. I do not have namenode
logs as I was hod for cluster allocation/deallocation with saving logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
