Sorry about the delay on this response, I was pulled onto other work. I figured out the problem. I needed to add the Cygwin\bin directory to my path so the Cygwin\bin\df.exe would execute. Apparently something changed in Hadoop 0.14 that caused the default configuration to execute that code path when it didn't prior to 0.14, but most people didn't notice because they already had it in their path or were running on Linux. It appears there was even an issue opened for it: https://issues.apache.org/jira/browse/HADOOP-1792
On Nov 9, 2007 9:20 AM, Enis Soztutar <[EMAIL PROTECTED]> wrote: > I checked for the usages of df in Hadoop for you. DF is used by many > parts of the system including dfs and mapred, but it will not run on a > node that does not run TaskTracker of Datanode. Could you please check > your configuration if you're missing smt. Can you confirm that you just > only submit the job from the windows machine? > > > Tim Gautier wrote: > > Thanks for the reply, but I think you missed my point. I've been > > running nutch and Hadoop through eclipse under Windows for several > > months now. It never called Linux shell commands before, now it is. > > Maybe it called df from some path I never hit, I don't know. What I > > do know is that I could do an entire Nutch crawl in eclipse on Windows > > with Hadoop 0.13 and I can't even inject with the latest version in > > trunk which is using Hadoop 0.15 because it calls df. I realize I can > > run it from the command line under cygwin, I just don't want to, I > > want to use eclipse like I have been for months. > > > > I feel like I'm missing something simple, but I can't figure out what > > it is. Anyone else have any ideas? > > > > On Nov 9, 2007 7:00 AM, Enis Soztutar <[EMAIL PROTECTED]> wrote: > > > >> Hadoop has been running df for a long time way before 0.13. You can run > >> hadoop under cygwin ın windows. Please refer to Hadoop's documentation. > >> > >> > >> Tim Gautier wrote: > >> > >>> I do my nutch development and debugging on a Windows XP machine before > >>> transferring my jar files to a Linux cluster for actual production > >>> runs. This has worked fine in the past, but there seems to be a > >>> problem now that we're using Hadoop .15. Now when I run the injector > >>> (presumably other classes as well) from Eclipse, Hadoop makes a shell > >>> command call to "df" and of course doesn't find it on a Windows > >>> machine so the job fails. There has to be a way around this so I can > >>> debug from Eclipse, anybody know what that might be? Configuration > >>> setting or something to tell it that I'm running on Windows? > >>> > >>> > >>> >
