piyushnarang commented on a change in pull request #8117: [FLINK-12115]
[filesystems]: Add support for AzureFS
URL: https://github.com/apache/flink/pull/8117#discussion_r280398656
##########
File path:
flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AbstractAzureFSFactory.java
##########
@@ -48,6 +55,33 @@ public void configure(Configuration config) {
public FileSystem create(URI fsUri) throws IOException {
checkNotNull(fsUri, "passed file system URI object should not
be null");
LOG.info("Trying to load and instantiate Azure File System");
- return new AzureFileSystem(fsUri, flinkConfig);
+ return new HadoopFileSystem(createInitializedAzureFS(fsUri,
flinkConfig));
+ }
+
+ // uri is of the form:
wasb(s)://[email protected]/testDir
+ private org.apache.hadoop.fs.FileSystem createInitializedAzureFS(URI
fsUri, Configuration flinkConfig) throws IOException {
+ org.apache.hadoop.conf.Configuration hadoopConfig =
HadoopUtils.getHadoopConfiguration(flinkConfig);
Review comment:
Yeah I can use the HadoopConfigLoader. It is a slightly more involved change
though as the config loader needs to be moved out of the s3-base module to a
new one and we also have other file system factories doing similar copying
which also need to be updated if we want to do things thoroughly.
If you're keen on using it here as part of this PR, I could kick start it by
creating a filesystem commons module, move the config loader there and use it
in this factory. wdyt?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services