[
https://issues.apache.org/jira/browse/HADOOP-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619403#comment-13619403
]
Sanjay Radia commented on HADOOP-9357:
--------------------------------------
Failing when authority-port is not specified was consciously done when
fileContext was implemented.
* A user wanting /tmp on the default file system should say "/tmp" and not
"hdfs:///tmp".
* While the notion of a default file system is very appropriate, the notion of
default authority per scheme does not make sense, especially if it is derived
from the default file system.
* In a multi-cluster environment, getting in the default authority/port from
the default file system can lead to accessing the files at the wrong HDFS
cluster/volume (each cluster has its own default and a user is allowed to
access the HDFS volumes in different HDFS clusters from within the same
application (as in distcp)).
* If you encourage the use of hdfs:///tmp then as you change your default file
system from hdfs to localfs, the app can suddenly fail. Being able to change
the local file system to different HDFS volumes/clusters, s3, localfs in a
fairly transparent fashion is a useful feature of Hadoop. For example, if /tmp
refers to the tmp on the default file system then things work transparently
when the the default file system is changed from hdfs://cluster1NN:37 to
hdfs://cluster2NN:55 to file:/// to s3fs etc.
Is this Jira being driven by customer apps that are breaking?
> Fallback to default authority if not specified in FileContext
> -------------------------------------------------------------
>
> Key: HADOOP-9357
> URL: https://issues.apache.org/jira/browse/HADOOP-9357
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Priority: Minor
> Fix For: 2.0.4-alpha
>
> Attachments: hadoop-9357-1.patch, hadoop-9357-2.patch,
> hadoop-9357-3.patch, hadoop-9357-testfixup.patch
>
>
> Currently, FileContext adheres rather strictly to RFC2396 when it comes to
> parsing absolute URIs (URIs with a scheme). If a user asks for a URI like
> "hdfs:///tmp", FileContext will error while FileSystem will add the authority
> of the default FS (e.g. turn it into "hdfs://defaultNN:port/tmp").
> This is technically correct, but FileSystem's behavior is nicer for users and
> okay based on 5.2.3 in the RFC, so lets do it in FileContext too:
> {noformat}
> For backwards
> compatibility, an implementation may work around such references
> by removing the scheme if it matches that of the base URI and the
> scheme is known to always use the syntax. The parser
> can then continue with the steps below for the remainder of the
> reference components. Validating parsers should mark such a
> misformed relative reference as an error.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
