Siyao Meng created HDFS-18001:
---------------------------------
Summary: RBF: FederationUtil.getJmx cannot parse a bracketed IPv6
NameNode web address
Key: HDFS-18001
URL: https://issues.apache.org/jira/browse/HDFS-18001
Project: Hadoop HDFS
Issue Type: Bug
Components: rbf
Reporter: Siyao Meng
Assignee: Siyao Meng
{{NamenodeHeartbeatService}} reads NameNode JMX through
{{FederationUtil.getJmx}} on every heartbeat
({{dfs.federation.router.namenode.heartbeat.jmx.interval}} defaults to 0). The
address passed in is the configured {{dfs.namenode.http-address.<ns>.<nn>}},
for example {{[2001:db8::10]:9870}}. {{getJmx}} splits it on {{:}}
([FederationUtil.java:78|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/FederationUtil.java#L78]):
{code:java}
if (webAddress.indexOf(":") > 0) {
String[] webAddressSplit = webAddress.split(":");
host = webAddressSplit[0];
port = Integer.parseInt(webAddressSplit[1]);
}
{code}
The host becomes {{[2001}} and {{Integer.parseInt("db8")}} throws
{{NumberFormatException}}. The {{catch (Exception)}} block logs an error and
returns null. This does not depend on reverse DNS.
As a result the FSNamesystem and NameNodeInfo values are never set,
{{NamenodeStatusReport.statsValid()}} stays false, and the Router stores no
membership stats for that NameNode: capacity, available space, files, blocks,
live and dead DataNodes. These feed the Router metrics and web UI, and
{{AvailableSpaceResolver}} reads the available space. The error is logged on
every heartbeat.
Verified by reading the code, and with a standalone copy of the parsing on JDK
21, which throws for {{[2001:db8::10]:9870}} and works for
{{nn1.example.com:9870}}. Not run on a Router. I did not trace what mount
points with the SPACE order do when stats are missing.
*Proposed fix:* parse the address with an IPv6-aware helper such as
{{NetUtils.createSocketAddrUnresolved}} or Guava {{HostAndPort.fromString}},
keeping the no-port case, and build the URL with {{new URL(scheme, host, port,
file)}}, which brackets an IPv6 host. Add a unit test with a bracketed IPv6
address.
POC:
https://github.com/smengcl/hadoop/commit/daab9184f1b0818161b3c53a50753aaaefb9f568
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]