[
https://issues.apache.org/jira/browse/FLINK-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17330152#comment-17330152
]
Matthias commented on FLINK-22414:
----------------------------------
Ok, sorry for not being clear on what I meant by "is used". I was referring to
"not being used in the Java code". I was wondering whether we could just access
{{env.hadoop.conf.dir}} in {{HadoopUtils.getHadoopConfiguration}} to have an
additional fallback there accessing {{env.hadoop.conf.dir}} and avoiding the
warning.
My understanding of the code is that {{HADOOP_CONF_DIR}} is set in
{{config.sh}} as [~chesnay] pointed out. But {{HADOOP_CONF_DIR}} is only added
to the
[INTERNAL_HADOOP_CLASSPATHS|https://github.com/apache/flink/blob/af681c63821c027bc7a233560f7765b686d0d244/flink-dist/src/main/flink-bin/bin/config.sh#L372]
but never exported as [~fly_in_gis] already mentioned. Hence, it is not picked
up by
[HadoopUtils.getHadoopConfiguration:113|https://github.com/apache/flink/blob/ba027e0b3e2cf950cd0bd7fb4f632a9c3449d9ad/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java#L113]
in that case producing the warning. Am I missing something here?
> Consider env.hadoop.conf.dir when looking for the hadoop configuration
> ----------------------------------------------------------------------
>
> Key: FLINK-22414
> URL: https://issues.apache.org/jira/browse/FLINK-22414
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Runtime / Configuration
> Affects Versions: 1.11.3, 1.12.2, 1.13.0, 1.14.0
> Reporter: Matthias
> Priority: Major
>
> The option of setting {{env.hadoop.conf.dir}} is allowed to be used as an
> alternative to setting the environment variable {{HADOOP_CONF_DIR}} based [on
> the
> documentation|https://ci.apache.org/projects/flink/flink-docs-stable/deployment/config.html#env-hadoop-conf-dir].
> But the variable is not checked when determining the configuration location
> in the code (see
> [HadoopUtils.getHadoopConfiguration|https://github.com/apache/flink/blob/ba027e0b3e2cf950cd0bd7fb4f632a9c3449d9ad/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java#L53]).
> This issue is about adding {{env.hadoop.conf.dir}} as an option to specify
> the Hadoop configuration directory.
> Initial proposal came from [this user mailing list
> thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Hadoop-config-on-docker-compose-td42967.html#a42994].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
