[ http://issues.apache.org/jira/browse/HADOOP-125?page=comments#action_12373917 ]
Doug Cutting commented on HADOOP-125: ------------------------------------- > Doesn't your fix break on unix if the user has a directory named "\\foo"? Perhaps. I did not test this case. I was just despately trying to get Nutch to work correctly standalone & pseudo-distributed on Windows and linux. There may still be bugs. Changing the global system directory caused strange behaviour on Windows, and also seems like a bad idea. Isn't the point of a default directory for job and fs instances to insulate them from the global system default directory? We do currently cache the LocalFs instance, which may be a bug, but changing the process's CWD affects other applications in the JVM (like even our own datanode & namenode) which might not expect the CWD to move under them. > Could we extend the FileSystem interface to add makeAbsolute? The long-term fix that I'd like to see is that we stop using java.io.File for file names. We could use String, we could use our own FileName class, or we could perhaps use java.net.URI. The advantage of URI is that it can also naturally include the namenode host and port. The disadvantage is that URI does not support tree operations like getParent(). I'll log a bug for this. > LocalFileSystem.makeAbsolute bug on Windows > ------------------------------------------- > > Key: HADOOP-125 > URL: http://issues.apache.org/jira/browse/HADOOP-125 > Project: Hadoop > Type: Bug > Components: fs > Environment: Windows > Reporter: paul sutter > Assignee: Doug Cutting > Priority: Minor > Fix For: 0.1.1, 0.2 > > LocalFileSystem.makeAbsolute() has a bug when running on Windows (which is > very useful for the development phase of a Hadoop task on one's laptop). > Problem: if a pathname such as /tmp/hadoop... is given in a config file, > when the jobconf file is created, it is put into the relative directory > named: currentdir/tmp/hadoop..., but when hadoop tries to open the file, it > looks in c:/tmp/hadoop..., and the job fails. > Cause: while Unix has two kinds of filespecs (relative and absolute), WIndows > actually has three: > (1) relative to current directory (subdir/file) > (2) relative to current disk (/dir/subdir/file) > (3) absolute (c:/dir/subdir/file) > So when a config file specifies a directory with what-is-on-unix an absolute > path (/tmp/hadoop...), the makeAbsolute() method will not work correctly. > Basically, File.isAbsolute() will return false for cases (1) and (2) above, > but true for case (3), which is not expected by the code below. > The solution would be to code explicit detection of all three casses for > Windows in the code below from LocalFileSystem: > private File makeAbsolute(File f) { > if (f.isAbsolute()) { > return f; > } else { > return new File(workingDir, f.toString()); > } > } > Im happy to explain if this explanation is confusing... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
