[
https://issues.apache.org/jira/browse/HADOOP-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029620#comment-14029620
]
Chris Nauroth commented on HADOOP-10663:
----------------------------------------
I'm not in favor of committing this patch. This is a continuation of comments
that I left on MAPREDUCE-5912 here:
https://issues.apache.org/jira/browse/MAPREDUCE-5912?focusedCommentId=14029597&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14029597
The issue raised in MAPREDUCE-5912 is that a change was made that started
looking for map outputs on the default file system instead of the local file
system. This causes problems on Windows if HDFS is the default file system,
because the path will have a colon due to the drive specifier, and a colon
isn't a valid character in an HDFS file name. As I said in my MAPREDUCE-5912
comment, I don't think that change was correct, even if it appears to be
working in non-Windows environments.
For the patch here, imagine a situation where the default file system is
neither the local file system nor HDFS, and that file system implementation
does allow colons in file names. With this patch, a {{Path}} with no scheme
and a path that "looks like" a Windows path with a drive spec can get misrouted
to local file system instead of the default file system. I admit this is
somewhat contrived, but it sure would create confusion if anyone ran into it.
I'd like to resolve this as won't fix. [~rusanu] and [~chris.douglas], does
this sound OK?
> Path.getFileSystem should identify Windows drive (c:) like files and return
> the localFS
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-10663
> URL: https://issues.apache.org/jira/browse/HADOOP-10663
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Remus Rusanu
> Assignee: Remus Rusanu
> Attachments: HADOOP-10663.1.patch
>
>
> While investigating the problem with MAPREDUCE-5912 I considered that the
> fact that asking a Windows specific Path to get its file system should not
> return the default FS, but the localFS. Specifically, I believe that
> Path.initialize should identify such names as belonging to the 'file://'
> scheme (when null) and so create appropriate URIs.
> I do not have the know-how to evaluate the bigger impact of such a change.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
