[
https://issues.apache.org/jira/browse/FLINK-10526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yan Yan closed FLINK-10526.
---------------------------
Resolution: Duplicate
> Hadoop FileSystem not initialized properly on Yarn
> --------------------------------------------------
>
> Key: FLINK-10526
> URL: https://issues.apache.org/jira/browse/FLINK-10526
> Project: Flink
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.4.0
> Reporter: Yan Yan
> Priority: Blocker
>
> When we were bumping Flink from 1.3.2 to 1.4.0, we noticed the
> HadoopFileSystem no longer reads flinkConfig.
> This prevents users to pass in custom HDFS configuration (e.g. core-site.xml)
> via _fs.hdfs.hadoopconf_.
> Specially it is due to the cached _HadoopFsFactory_ is initialized with an
> dummy Configuration
> ([[code|https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/core/fs/FileSystem.java#L387]|https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/core/fs/FileSystem.java#L387]),
> which prevents future _flinkConfig_ getting passed in
> ([[code|https://github.com/apache/flink/blob/master/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java#L84]|https://github.com/apache/flink/blob/master/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java#L84]).
> We have verified by calling _FileSystem.initialize(flinkConfig)_ explicitly
> in _YarnApplicationMasterRunner.run_ solve the issue.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
