[
https://issues.apache.org/jira/browse/FLINK-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17330474#comment-17330474
]
Chesnay Schepler commented on FLINK-22414:
------------------------------------------
[~f.pompermaier] yes, config.sh does not pick up FLINK_PROPERTIES, this is
something handled by the docker scripts, which merges it into the flink
configuration.
[~fly_in_gis] Indeed, we never export HADOOP_CONF_DIR, but I wonder whether
that would really be a right solution. Presumably there is a reason why the
user does not export HADOOP_CONF_DIR directly, so let's maybe not mess with
that.
I would've expected that, if a core-site.xml file is present on the classpath
(which is what config.sh does), that it is automatically picked up by Hadoop.
If that is not the case, then we could either also evaluated the option in
HadoopUtils (which I'd rather not because it implies spreading accesses across
bash and java code), or add the core-site.xml from the classpath as a default
resource to the created configuration.
Or, maybe we should just discourage using this option and leave it as is; in
the reported issue the user is easily able to set HADOOP_CONF_DIR.
> Consider env.hadoop.conf.dir when looking for the hadoop configuration
> ----------------------------------------------------------------------
>
> Key: FLINK-22414
> URL: https://issues.apache.org/jira/browse/FLINK-22414
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Runtime / Configuration
> Affects Versions: 1.11.3, 1.12.2, 1.13.0, 1.14.0
> Reporter: Matthias
> Priority: Major
>
> The option of setting {{env.hadoop.conf.dir}} is allowed to be used as an
> alternative to setting the environment variable {{HADOOP_CONF_DIR}} based [on
> the
> documentation|https://ci.apache.org/projects/flink/flink-docs-stable/deployment/config.html#env-hadoop-conf-dir].
> But the variable is not checked when determining the configuration location
> in the code (see
> [HadoopUtils.getHadoopConfiguration|https://github.com/apache/flink/blob/ba027e0b3e2cf950cd0bd7fb4f632a9c3449d9ad/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java#L53]).
> This issue is about adding {{env.hadoop.conf.dir}} as an option to specify
> the Hadoop configuration directory.
> Initial proposal came from [this user mailing list
> thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Hadoop-config-on-docker-compose-td42967.html#a42994].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
