[
https://issues.apache.org/jira/browse/HDFS-197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212763#comment-13212763
]
Carlos Duque commented on HDFS-197:
-----------------------------------
I found out that the "du.exe" version that is ran by default is the one located
at "%SystemRoot%\system32\" and the output is very different from the one in
Cygwin's "bin" directory, actually it is not the expected output because the
first line is blank.
The options I came up with are:
1. Setting up the %PATH% so that Cygwin's "bin" directory is read before
"%SystemRoot%\system32" (dangerous for the whole system IMO)
2. Overwriting "du.exe" with Cygwin's version (dangerous for part of the system
IMO)
3. Somehow setting in Hadoop's config files the path from where to read the
right binaries, in this case "du.exe"
I don't know if option 3 is available right now (I'm new to Hadoop), but I hope
this info helps solve the problem.
> "du" fails on Cygwin
> --------------------
>
> Key: HDFS-197
> URL: https://issues.apache.org/jira/browse/HDFS-197
> Project: Hadoop HDFS
> Issue Type: Bug
> Environment: Windows + Cygwin
> Reporter: Kohsuke Kawaguchi
> Attachments: HADOOP-5486
>
>
> When I try to run a datanode on Windows, I get the following exception:
> {noformat}
> java.io.IOException: Expecting a line not the end of stream
> at org.apache.hadoop.fs.DU.parseExecResult(DU.java:181)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:179)
> at org.apache.hadoop.util.Shell.run(Shell.java:134)
> at org.apache.hadoop.fs.DU.<init>(DU.java:53)
> at org.apache.hadoop.fs.DU.<init>(DU.java:63)
> at
> org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.<init>(FSDataset.java:325)
> at
> org.apache.hadoop.hdfs.server.datanode.FSDataset.<init>(FSDataset.java:681)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:291)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1238)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1193)
> {noformat}
> This is because Hadoop execs "du -sk C:\tmp\hadoop-SYSTEM\dfs\data" with a
> Windows path representation, which cygwin du doesn't understand.
> {noformat}
> C:\hudson>du -sk C:\tmp\hadoop-SYSTEM\dfs\data
> du -sk C:\tmp\hadoop-SYSTEM\dfs\data
> du: cannot access `C:\\tmp\\hadoop-SYSTEM\\dfs\\data': No such file or
> directory
> {noformat}
> For this to work correctly, Hadoop would have to run cygpath first to get a
> Unix path representation, then to call DU.
> Also, I had to use the debugger to get this information. Shell.runCommand
> should catch IOException from parseExecResult and add the buffered stderr to
> simplify the error diagnostics.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
