[
https://issues.apache.org/jira/browse/DRILL-5365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497294#comment-16497294
]
ASF GitHub Bot commented on DRILL-5365:
---------------------------------------
ilooner closed pull request #796: DRILL-5365: DrillFileSystem setConf in
constructor. DrillFileSystem c…
URL: https://github.com/apache/drill/pull/796
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java
index e03cf22223..9ce7c6d68d 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillFileSystem.java
@@ -89,22 +89,36 @@ public DrillFileSystem(Configuration fsConf) throws
IOException {
}
public DrillFileSystem(Configuration fsConf, OperatorStats operatorStats)
throws IOException {
- this.underlyingFs = FileSystem.get(fsConf);
+ this(fsConf, URI.create(fsConf.getRaw(FS_DEFAULT_NAME_KEY)),
operatorStats);
+ }
+
+ public DrillFileSystem(Configuration fsConf, URI Uri, OperatorStats
operatorStats) throws IOException {
+ this.underlyingFs = FileSystem.get(Uri, fsConf);
+ logger.trace("Configuration for the DrillFileSystem " +
fsConf.getRaw(FS_DEFAULT_NAME_KEY) +
+ ", underlyingFs: " + this.underlyingFs.getUri());
this.codecFactory = new CompressionCodecFactory(fsConf);
this.operatorStats = operatorStats;
+ setConf(fsConf);
}
@Override
public void setConf(Configuration conf) {
// Guard against setConf(null) call that is called as part of superclass
constructor (Configured) of the
// DrillFileSystem, at which point underlyingFs is null.
- if (conf != null && underlyingFs != null) {
- underlyingFs.setConf(conf);
+ if(conf != null) {
+ super.setConf(conf);
+ if (underlyingFs != null && underlyingFs.getConf() == null) {
+ underlyingFs.setConf(conf);
+ }
+
}
}
@Override
public Configuration getConf() {
+ if (super.getConf() != null) {
+ return super.getConf();
+ }
return underlyingFs.getConf();
}
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
index 7f2a9c16a7..b5f087117b 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
@@ -74,6 +74,7 @@ public FileSystemPlugin(FileSystemConfig config,
DrillbitContext context, String
fsConf.set(s, config.config.get(s));
}
}
+ fsConf.set("fs.default.name", config.connection);
fsConf.set(FileSystem.FS_DEFAULT_NAME_KEY, config.connection);
fsConf.set("fs.classpath.impl", ClassPathFileSystem.class.getName());
fsConf.set("fs.drill-local.impl",
LocalSyncableFileSystem.class.getName());
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> FileNotFoundException when reading a parquet file
> -------------------------------------------------
>
> Key: DRILL-5365
> URL: https://issues.apache.org/jira/browse/DRILL-5365
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
> Affects Versions: 1.10.0
> Reporter: Chun Chang
> Assignee: Timothy Farkas
> Priority: Major
> Fix For: 1.14.0
>
>
> The parquet file is generated through the following CTAS.
> To reproduce the issue: 1) two or more nodes cluster; 2) enable
> impersonation; 3) set "fs.default.name": "file:///" in hive storage plugin;
> 4) restart drillbits; 5) as a regular user, on node A, drop the table/file;
> 6) ctas from a large enough hive table as source to recreate the table/file;
> 7) query the table from node A should work; 8) query from node B as same user
> should reproduce the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
