"du" fails on Cygwin
--------------------
Key: HADOOP-5496
URL: https://issues.apache.org/jira/browse/HADOOP-5496
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.19.1
Environment: Windows + Cygwin
Reporter: Kohsuke Kawaguchi
When I try to run a datanode on Windows, I get the following exception:
{noformat}
java.io.IOException: Expecting a line not the end of stream
at org.apache.hadoop.fs.DU.parseExecResult(DU.java:181)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:179)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.fs.DU.<init>(DU.java:53)
at org.apache.hadoop.fs.DU.<init>(DU.java:63)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.<init>(FSDataset.java:325)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset.<init>(FSDataset.java:681)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:291)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1238)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1193)
{noformat}
This is because Hadoop execs "du -sk C:\tmp\hadoop-SYSTEM\dfs\data" with a
Windows path representation, which cygwin du doesn't understand.
{noformat}
C:\hudson>du -sk C:\tmp\hadoop-SYSTEM\dfs\data
du -sk C:\tmp\hadoop-SYSTEM\dfs\data
du: cannot access `C:\\tmp\\hadoop-SYSTEM\\dfs\\data': No such file or directory
{noformat}
For this to work correctly, Hadoop would have to run cygpath first to get a
Unix path representation, then to call DU.
Also, I had to use the debugger to get this information. Shell.runCommand
should catch IOException from parseExecResult and add the buffered stderr to
simplify the error diagnostics.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
