john lilley created HDFS-8353:
---------------------------------
Summary: FileSystem.resolvePath() fails for Windows UNC paths
Key: HDFS-8353
URL: https://issues.apache.org/jira/browse/HDFS-8353
Project: Hadoop HDFS
Issue Type: Bug
Components: HDFS
Affects Versions: 2.6.0, 2.4.0
Environment: Windows 8, x64
Java 1.7
Reporter: john lilley
FileSystem.resolvePath() fails with Windows UNC path. This has a knock-on
effect with Parquet file access in local filesystem, because Parquet has no
open-using-stream API that we've been able to find. Take this simple test:
public class Scratch {
public static void main(String[] args) {
// Note that this path must exist
java.net.URI uriWithAuth = java.net.URI.create("file://host/share/file");
try {
FileSystem fs = FileSystem.get(uriWithAuth, new Configuration());
fs.resolvePath(new Path(uriWithAuth));
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
The resolvePath() call will fail in FileSystem.checkPath():
protected void checkPath(Path path) {
URI uri = path.toUri();
String thatScheme = uri.getScheme();
if (thatScheme == null) // fs is relative
return;
URI thisUri = getCanonicalUri();
String thisScheme = thisUri.getScheme();
//authority and scheme are not case sensitive
if (thisScheme.equalsIgnoreCase(thatScheme)) {// schemes match
String thisAuthority = thisUri.getAuthority();
String thatAuthority = uri.getAuthority();
if (thatAuthority == null && // path's authority is null
thisAuthority != null) { // fs has an authority
URI defaultUri = getDefaultUri(getConf());
if (thisScheme.equalsIgnoreCase(defaultUri.getScheme())) {
uri = defaultUri; // schemes match, so use this uri instead
} else {
uri = null; // can't determine auth of the path
}
}
if (uri != null) {
// canonicalize uri before comparing with this fs
uri = canonicalizeUri(uri);
thatAuthority = uri.getAuthority();
if (thisAuthority == thatAuthority || // authorities match
(thisAuthority != null &&
thisAuthority.equalsIgnoreCase(thatAuthority)))
return;
}
}
throw new IllegalArgumentException("Wrong FS: "+path+
", expected: "+this.getUri());
}
The problem is that thisAuthority is null, and thatAuthority gets "host". There
is no logic for dealing with that case. In fact, this method seems broken in
several ways. There are at least two problems:
-- For UNC paths like file://host/share/... , the authority does not need to
match, at least for Windows UNC paths. All of these paths are the same file
system: "file:///F:/folder/file", "file://host1/share/file",
"file://host2/share/file".
-- The test thisAuthority == thatAuthority violates Java 101. It should be
thisAuthority.equals(thatAuthority)
-- hostnames are case-independent, so I think that the authority comparison
should also be case-insensitive, at least for UNC paths.
-- I don't see any attempt to resolve hostnames to IP addresses, but that may
simply be beyond the scope of this method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
