[
https://issues.apache.org/jira/browse/HADOOP-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619485#comment-13619485
]
Andrew Wang commented on HADOOP-9357:
-------------------------------------
Hi Sanjay,
Thanks for taking a look, I understand and agree with most of your points. I
created this JIRA after some end user confusion when porting an app from
FileSystem to FileContext. I felt that URIs like {{hdfs:///tmp}} should be
discouraged but supported, since they are supported by FileSystem, and this
case is specifically mentioned in the RFC as an okay fallback.
A couple potentially clarifying points:
bq. A user wanting /tmp on the default file system should say "/tmp" and not
"hdfs:///tmp".
This code triggers when the schema matches the schema of the default FS.
{{hdfs:///tmp}} will only work if the default FS is an HDFS instance.
bq. While the notion of a default file system is very appropriate, the notion
of default authority per scheme does not make sense, especially if it is
derived from the default file system.
There is not a default authority per scheme, as per the previous.
bq. In a multi-cluster environment, getting in the default authority/port from
the default file system can lead to accessing the files at the wrong HDFS
cluster/volume...
bq. If you encourage the use of hdfs:///tmp then as you change your default
file system from hdfs to localfs...
I agree there could be ambiguity if you have multiple FS with the same schema,
or change the default FS.
Moving toward a meta question, we already have all these problems with
FileSystem, which is the dominant interface to HDFS. Have we encountered these
problems in FileSystem with ambiguity and changing the default FS? I at least
haven't heard of any, but I did see the mail that led to this JIRA.
> Fallback to default authority if not specified in FileContext
> -------------------------------------------------------------
>
> Key: HADOOP-9357
> URL: https://issues.apache.org/jira/browse/HADOOP-9357
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Priority: Minor
> Fix For: 2.0.4-alpha
>
> Attachments: hadoop-9357-1.patch, hadoop-9357-2.patch,
> hadoop-9357-3.patch, hadoop-9357-testfixup.patch
>
>
> Currently, FileContext adheres rather strictly to RFC2396 when it comes to
> parsing absolute URIs (URIs with a scheme). If a user asks for a URI like
> "hdfs:///tmp", FileContext will error while FileSystem will add the authority
> of the default FS (e.g. turn it into "hdfs://defaultNN:port/tmp").
> This is technically correct, but FileSystem's behavior is nicer for users and
> okay based on 5.2.3 in the RFC, so lets do it in FileContext too:
> {noformat}
> For backwards
> compatibility, an implementation may work around such references
> by removing the scheme if it matches that of the base URI and the
> scheme is known to always use the syntax. The parser
> can then continue with the steps below for the remainder of the
> reference components. Validating parsers should mark such a
> misformed relative reference as an error.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
