[
https://issues.apache.org/jira/browse/FLINK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-25413:
-----------------------------------
Labels: auto-deprioritized-major pull-request-available (was:
pull-request-available stale-major)
Priority: Minor (was: Major)
This issue was labeled "stale-major" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Major, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Use append dfs.nameservices hadoop config to replace override
> --------------------------------------------------------------
>
> Key: FLINK-25413
> URL: https://issues.apache.org/jira/browse/FLINK-25413
> Project: Flink
> Issue Type: Improvement
> Reporter: qiunan
> Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available
>
> In FLINK-16005[flink-yarn] Support yarn and hadoop config override.
> In flink-conf.yaml
> flink.hadoop.dfs.namenode.rpc-address.nameservice1.nn1: bigdata1:8020
> flink.hadoop.dfs.namenode.rpc-address.nameservice1.nn2: bigdata2:8020
> flink.hadoop.dfs.client.failover.proxy.provider.nameservice1:
> org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> flink.hadoop.dfs.ha.namenodes.nameservice1: nn1,nn2
> flink.hadoop.dfs.nameservices: nameservice1
> code：
> {code:java}
> // Approach 4: Flink configuration
> // add all configuration key with prefix 'flink.hadoop.' in flink conf to
> hadoop conf
> for (String key : flinkConfiguration.keySet()) {
> for (String prefix : FLINK_CONFIG_PREFIXES) {
> if (key.startsWith(prefix)) {
> String newKey = key.substring(prefix.length());
> String value = flinkConfiguration.getString(key, null);
> result.set(newKey, value);
> LOG.debug(
> "Adding Flink config entry for {} as {}={} to Hadoop
> config",
> key,
> newKey,
> value);
> foundHadoopConfiguration = true;
> }
> }
> } {code}
> If my HADOOP_CONF_DIR hdfs-site.xml have dfs.nameservices: nameservice2, see
> the code logic this config will be override. I think this config should not
> be override append will be better. if override we should add all config, but
> we have many clusters in production, it is impossible to configure all
> configurations in flink-conf.yaml.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
