[
https://issues.apache.org/jira/browse/HADOOP-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley resolved HADOOP-528.
----------------------------------
Resolution: Cannot Reproduce
Assignee: (was: Owen O'Malley)
I haven't seen this recently and I suspect it was the race condition in
creating the directories that we used to have with mkdirs calling exists and
then mkdir.
> Some hadoop distcp task fails with dfs.client.buffer.dir not found error
> while the property exists.
> ---------------------------------------------------------------------------------------------------
>
> Key: HADOOP-528
> URL: https://issues.apache.org/jira/browse/HADOOP-528
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.6.0
> Reporter: Sanjay Dahiya
>
> On some nodes hadoop distcp fails with folowing error. The property
> dfs.client.buffer.dir is defined in hadoop-site.xml and the corresponding
> directory is also created on the node by framework. Same configuration works
> on all other nodes.
> It could be a timing issue where someone is trying to read the dir before its
> created on local disk, I am not sure of the cause.
> java.io.IOException: No valid local directories in property:
> dfs.client.buffer.dir at
> org.apache.hadoop.conf.Configuration.getFile(Configuration.java:405) at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:780)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:770) at
> org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:276) at
> org.apache.hadoop.dfs.DistributedFileSystem.createRaw(DistributedFileSystem.java:104)
> at
> org.apache.hadoop.fs.FSDataOutputStream$Summer.(FSDataOutputStream.java:56)
> at
> org.apache.hadoop.fs.FSDataOutputStream$Summer.(FSDataOutputStream.java:45)
> at org.apache.hadoop.fs.FSDataOutputStream.(FSDataOutputStream.java:146) at
> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:270) at
> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:177) at
> org.apache.hadoop.util.CopyFiles$DFSCopyFilesMapper.copy(CopyFiles.java:186)
> at
> org.apache.hadoop.util.CopyFiles$DFSCopyFilesMapper.map(CopyFiles.java:391)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:210) at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1066)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
