[
https://issues.apache.org/jira/browse/HDDS-17135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17135:
------------------------------
Fix Version/s: 2.3.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon
> datanode JMX URLs
> ----------------------------------------------------------------------------------------
>
> Key: HDDS-17135
> URL: https://issues.apache.org/jira/browse/HDDS-17135
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.0
>
> Attachments: HDDS-17135.001.patch, HDDS-17135.002.patch
>
>
> Several production paths still pass a host that can be a bare IPv6 literal to
> {{NetUtils.createSocketAddr(String target, int defaultPort)}}, or format it
> into a URL without brackets.
> In Hadoop 3.4.3, {{createSocketAddr(target, defaultPort)}} parses {{target}}
> as a URI authority ({{"dummyscheme://" + target}}). For a bare literal such
> as {{2001:db8::10}} the URI host is null and the call throws
> {{IllegalArgumentException: Does not contain a valid host:port authority}}.
> Bare literals reach these paths because SCM records a datanode's IP with
> {{InetAddress.getHostAddress()}}. It also records the datanode hostname with
> {{getHostName()}}, which returns the literal when there is no reverse DNS.
> Likewise {{NodeDetails}} takes its host address from
> {{InetSocketAddress.getHostName()}}.
> h3. Affected paths
> # {{NodeDetails.getRpcAddress()}} (hdds-common):
> {{NetUtils.createSocketAddr(hostAddress, rpcPort)}}.
> {{OMNodeDetails.getFromProtobuf}} and the OM bootstrap handler build node
> details from the remote OM's host address. When that address is a bare
> literal, {{OzoneManager.checkRemoteOMConfig}} and {{addOMToRatisRing}} ->
> {{getRatisHostPortStr()}} throw an unchecked {{IllegalArgumentException}}.
> # {{SCMHANodeDetails.loadSCMHAConfig}} (server-scm):
> {{NetUtils.createSocketAddr(rpcAddrStr, ratisPort)}}.
> {{validateAdvertisedHost}} and {{SCMNodeInfo}} accept an unbracketed
> {{ozone.scm.address.<serviceId>.<nodeId>}} such as {{2001:db8::2}}, but this
> call throws for it, so SCM HA fails to start.
> # {{XceiverClientManager.getPipelineCacheKey}} and the
> {{XceiverClientShortCircuit}} constructor (hdds-client):
> {{NetUtils.createSocketAddr(dn.getIpAddress(), port)}}. With
> {{ozone.client.read.short-circuit=true}}, every non-EC read from an IPv6
> datanode fails.
> # {{DataNodeMetricsCollectionTask.getJmxMetricsUrl}} (Recon):
> {{String.format("%s://%s:%d/jmx", ...)}} with the datanode hostname. The URL
> is malformed for a literal host, so the datanode's pending-deletion metrics
> are reported as -1.
> # {{DiskBalancerSubCommandUtil.getSingleNodeDiskBalancerProxy}} (cli-admin):
> it treats any {{:}} in the address as "port given". So {{[2001:db8::1]}} and
> a bare {{2001:db8::1}} throw instead of using the default client port. Only
> {{[2001:db8::1]:19864}} works.
> # {{ozone local}} (ozone-tools): {{LocalOzoneCluster.address()}} and
> {{getReconEndpoint()}} build {{host + ":" + port}}, and {{OzoneLocal}} prints
> the SCM and OM addresses the same way. With {{--host ::1}}, every service
> address becomes {{::1:9860}}. The advertised-address validation from
> HDDS-16308 then rejects these values.
> h3. Fix
> Combine the host and port with {{HddsUtils.getHostPortString(host, port)}},
> and parse the DiskBalancer address with {{HddsUtils.getHostName}} /
> {{getHostPort}}, which brackets IPv6 literals and accepts DNS names, IPv4
> literals and already-bracketed input. This is the same pattern used by
> HDDS-15773 and HDDS-16831.
> h3. Testing
> Unit tests cover:
> * {{OMNodeDetails}} with an unbracketed IPv6 host address
> * SCM HA configuration with unbracketed and bracketed IPv6 peer addresses
> * the Recon datanode JMX URL for a DNS name and an IPv6 literal over HTTP and
> HTTPS
> * DiskBalancer address parsing for IPv4, bare IPv6 and bracketed IPv6, with
> and without a port
> * {{ozone local}} service addresses, Recon endpoint and startup summary with
> {{--host ::1}}
> Each new test fails without the fix.
> The IPv6-only acceptance test (HDDS-16969) does not catch these because
> compose resolves nodes by DNS name.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
