[
https://issues.apache.org/jira/browse/HDFS-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiaojunxiang updated HDFS-17460:
--------------------------------
Description:
In HDFS, `dfs.internal.nameservices` is a server-side configuration, while
`dfs.nameservices` is both a server-side and HDFSClient-side configuration.
The internal configuration implies it's for internal systems.
Now on the DataNode and JournalNode side, the priority of
dfs.internal.nameservices configuration is higher than `dfs.nameservices`,
which makes sense. {color:#FF0000}*However,*{color} on the ZKFC + NameNode
side, internal configuration isn't considered.
In such a scenario, when I enable RBF, the NameNode side expects
`dfs.nameservices`=ns1,ns2 (It now ignoring `dfs.internal.nameservices`),
{color:#FF0000}while {color}the HDFSClient side expects
`dfs.nameservices`=ns1,ns2,nsRouter (It now ignoring
`dfs.internal.nameservices`).
{color:#FF0000}This means they can't use the same configuration, It would be
too inconvenient{color}
If I unify all {color:#FF0000}server-side{color} configurations to use
`dfs.internal.nameservices`, meaning using the value of
`dfs.internal.nameservices` to {color:#FF0000}override
{color}`dfs.nameservices` during process startup,
{color:#FF0000}it brings the following benefits{color}:
1、Both server and HDFSClient can {color:#FF0000}use the same
configuration{color}: `dfs.nameservices`=ns1,ns2,nsRouter,
`dfs.internal.nameservices`=ns1,ns2.
2、Previously, the priority of `dfs.internal.nameservices` over
`dfs.nameservices` was only considered in DataNode and JournalNode
configurations, but NameNode didn't consider dfs.internal.nameservices. This
inconsistency in internal services doesn't look good and can be avoided. And
now, by unifying them, {color:#FF0000}it better aligns with the original
semantic meaning of the word "internal" as it pertains to internal
systems.{color}
was:
In HDFS, dfs.internal.nameservices is a server-side configuration, while
dfs.nameservices is both a server-side and client-side configuration.
The internal configuration implies it's for internal systems.
This means on the DataNode and JournalNode side, the priority of
dfs.internal.nameservices configuration is higher than dfs.nameservices, which
makes sense.
However, on the ZKFC + NameNode side, internal configuration isn't considered.
In such a scenario, when I enable RBF, the NameNode side expects
dfs.nameservices=ns1,ns2 (ignoring dfs.internal.nameservices),
while the HDFSClient side expects dfs.nameservices=ns1,ns2,nsRouter (ignoring
dfs.internal.nameservices).
This means they can't use the same configuration.
If I unify all server-side configurations to use dfs.internal.nameservices,
meaning using the value of dfs.internal.nameservices to override
dfs.nameservices during process startup,
it brings the following benefits:
1、Both server and client can use the same configuration:
dfs.nameservices=ns1,ns2,nsRouter, dfs.internal.nameservices=ns1,ns2.
2、Previously, the priority of dfs.internal.nameservices over dfs.nameservices
was only considered in DataNode and JournalNode configurations, but NameNode
didn't consider dfs.internal.nameservices. This inconsistency in internal
services doesn't look good and can be avoided. And now, by unifying them, it
better aligns with the original semantic meaning of the word "internal" as it
pertains to internal systems.
> Make All HDFS Server side component uniform use the value of
> `dfs.internal.nameservices` to override `dfs.nameservices`
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17460
> URL: https://issues.apache.org/jira/browse/HDFS-17460
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, rbf
> Reporter: xiaojunxiang
> Priority: Major
>
> In HDFS, `dfs.internal.nameservices` is a server-side configuration, while
> `dfs.nameservices` is both a server-side and HDFSClient-side configuration.
>
> The internal configuration implies it's for internal systems.
> Now on the DataNode and JournalNode side, the priority of
> dfs.internal.nameservices configuration is higher than `dfs.nameservices`,
> which makes sense. {color:#FF0000}*However,*{color} on the ZKFC + NameNode
> side, internal configuration isn't considered.
>
> In such a scenario, when I enable RBF, the NameNode side expects
> `dfs.nameservices`=ns1,ns2 (It now ignoring `dfs.internal.nameservices`),
> {color:#FF0000}while {color}the HDFSClient side expects
> `dfs.nameservices`=ns1,ns2,nsRouter (It now ignoring
> `dfs.internal.nameservices`).
> {color:#FF0000}This means they can't use the same configuration, It would be
> too inconvenient{color}
>
> If I unify all {color:#FF0000}server-side{color} configurations to use
> `dfs.internal.nameservices`, meaning using the value of
> `dfs.internal.nameservices` to {color:#FF0000}override
> {color}`dfs.nameservices` during process startup,
> {color:#FF0000}it brings the following benefits{color}:
> 1、Both server and HDFSClient can {color:#FF0000}use the same
> configuration{color}: `dfs.nameservices`=ns1,ns2,nsRouter,
> `dfs.internal.nameservices`=ns1,ns2.
> 2、Previously, the priority of `dfs.internal.nameservices` over
> `dfs.nameservices` was only considered in DataNode and JournalNode
> configurations, but NameNode didn't consider dfs.internal.nameservices. This
> inconsistency in internal services doesn't look good and can be avoided. And
> now, by unifying them, {color:#FF0000}it better aligns with the original
> semantic meaning of the word "internal" as it pertains to internal
> systems.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
