[
https://issues.apache.org/jira/browse/FLINK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-25413:
-----------------------------------
Labels: pull-request-available stale-major (was: pull-request-available)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Major but is unassigned and neither itself nor its Sub-Tasks have been updated
for 60 days. I have gone ahead and added a "stale-major" to the issue". If this
ticket is a Major, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Use append dfs.nameservices hadoop config to replace override
> --------------------------------------------------------------
>
> Key: FLINK-25413
> URL: https://issues.apache.org/jira/browse/FLINK-25413
> Project: Flink
> Issue Type: Improvement
> Reporter: qiunan
> Priority: Major
> Labels: pull-request-available, stale-major
>
> In FLINK-16005[flink-yarn] Support yarn and hadoop config override.
> In flink-conf.yaml
> flink.hadoop.dfs.namenode.rpc-address.nameservice1.nn1: bigdata1:8020
> flink.hadoop.dfs.namenode.rpc-address.nameservice1.nn2: bigdata2:8020
> flink.hadoop.dfs.client.failover.proxy.provider.nameservice1:
> org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> flink.hadoop.dfs.ha.namenodes.nameservice1: nn1,nn2
> flink.hadoop.dfs.nameservices: nameservice1
> code：
> {code:java}
> // Approach 4: Flink configuration
> // add all configuration key with prefix 'flink.hadoop.' in flink conf to
> hadoop conf
> for (String key : flinkConfiguration.keySet()) {
> for (String prefix : FLINK_CONFIG_PREFIXES) {
> if (key.startsWith(prefix)) {
> String newKey = key.substring(prefix.length());
> String value = flinkConfiguration.getString(key, null);
> result.set(newKey, value);
> LOG.debug(
> "Adding Flink config entry for {} as {}={} to Hadoop
> config",
> key,
> newKey,
> value);
> foundHadoopConfiguration = true;
> }
> }
> } {code}
> If my HADOOP_CONF_DIR hdfs-site.xml have dfs.nameservices: nameservice2, see
> the code logic this config will be override. I think this config should not
> be override append will be better. if override we should add all config, but
> we have many clusters in production, it is impossible to configure all
> configurations in flink-conf.yaml.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
