[
https://issues.apache.org/jira/browse/HDFS-8353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated HDFS-8353:
--------------------------------------
Component/s: (was: HDFS)
fs
> FileSystem.resolvePath() fails for Windows UNC paths
> ----------------------------------------------------
>
> Key: HDFS-8353
> URL: https://issues.apache.org/jira/browse/HDFS-8353
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.4.0, 2.6.0
> Environment: Windows 8, x64
> Java 1.7
> Reporter: john lilley
>
> FileSystem.resolvePath() fails with Windows UNC path. This has a knock-on
> effect with Parquet file access in local filesystem, because Parquet has no
> open-using-stream API that we've been able to find. Take this simple test:
> public class Scratch {
> public static void main(String[] args) {
> // Note that this path must exist
> java.net.URI uriWithAuth = java.net.URI.create("file://host/share/file");
> try {
> FileSystem fs = FileSystem.get(uriWithAuth, new Configuration());
> fs.resolvePath(new Path(uriWithAuth));
> } catch (Exception ex) {
> ex.printStackTrace();
> }
> }
> }
> The resolvePath() call will fail in FileSystem.checkPath():
> protected void checkPath(Path path) {
> URI uri = path.toUri();
> String thatScheme = uri.getScheme();
> if (thatScheme == null) // fs is relative
> return;
> URI thisUri = getCanonicalUri();
> String thisScheme = thisUri.getScheme();
> //authority and scheme are not case sensitive
> if (thisScheme.equalsIgnoreCase(thatScheme)) {// schemes match
> String thisAuthority = thisUri.getAuthority();
> String thatAuthority = uri.getAuthority();
> if (thatAuthority == null && // path's authority is null
> thisAuthority != null) { // fs has an authority
> URI defaultUri = getDefaultUri(getConf());
> if (thisScheme.equalsIgnoreCase(defaultUri.getScheme())) {
> uri = defaultUri; // schemes match, so use this uri instead
> } else {
> uri = null; // can't determine auth of the path
> }
> }
> if (uri != null) {
> // canonicalize uri before comparing with this fs
> uri = canonicalizeUri(uri);
> thatAuthority = uri.getAuthority();
> if (thisAuthority == thatAuthority || // authorities match
> (thisAuthority != null &&
> thisAuthority.equalsIgnoreCase(thatAuthority)))
> return;
> }
> }
> throw new IllegalArgumentException("Wrong FS: "+path+
> ", expected: "+this.getUri());
> }
> The problem is that thisAuthority is null, and thatAuthority gets "host".
> There is no logic for dealing with that case. In fact, this method seems
> broken in several ways. There are at least two problems:
> -- For UNC paths like file://host/share/... , the authority does not need to
> match, at least for Windows UNC paths. All of these paths are the same file
> system: "file:///F:/folder/file", "file://host1/share/file",
> "file://host2/share/file".
> -- The test thisAuthority == thatAuthority violates Java 101. It should be
> thisAuthority.equals(thatAuthority)
> -- hostnames are case-independent, so I think that the authority comparison
> should also be case-insensitive, at least for UNC paths.
> -- I don't see any attempt to resolve hostnames to IP addresses, but that may
> simply be beyond the scope of this method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
