Siyao Meng created HADOOP-20028:
-----------------------------------
Summary: SecurityUtil.getByName rejects bracketed IPv6 literals
when token service use_ip is false
Key: HADOOP-20028
URL: https://issues.apache.org/jira/browse/HADOOP-20028
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Siyao Meng
Assignee: Siyao Meng
{{SecurityUtil.getByName}} gives different results for a bracketed IPv6 literal
such as {{[2001:db8::1]}} depending on {{hadoop.security.token.service.use_ip}}:
* {{use_ip=true}} (default): {{StandardHostResolver}} calls
{{InetAddress.getByName}}, which accepts the brackets and returns the address.
* {{use_ip=false}}: {{QualifiedHostResolver.resolve}} checks
{{InetAddresses.isInetAddress(host)}}, which is false for the bracketed form.
The literal then goes down the host name path ({{getByNameWithSearch}},
{{getByExactName}}), where the JDK rejects names like
{{[2001:db8::1].example.com.}} as invalid IPv6 literals, and {{resolve}} throws
{{UnknownHostException}}.
[SecurityUtil.java:711|https://github.com/apache/hadoop/blob/9ede2e95c6fb577d7ae8664833d5e151cbfc47c9/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java#L711]
{{URI.getHost()}} returns IPv6 hosts with brackets, so callers that pass it on
hit this. Examples on trunk:
* {{DelegationTokenSelector.selectToken}} calls
{{NetUtils.createSocketAddrForHost(nnUri.getHost(), nnRpcPort)}}. With
{{use_ip=false}} this returns an unresolved address for
{{hdfs://[2001:db8::1]:8020}}.
* {{NetUtils.getCanonicalUri}} passes {{uri.getHost()}} to
{{canonicalizeHost}}, which leaves the host as is when the lookup fails.
Failures are not cached, so the lookup is repeated on every call.
HADOOP-19958 made {{NetUtils.createSocketAddr}} strip the brackets before
resolving, which covers the most common path. I have not confirmed a
user-visible failure from the remaining callers; the token service names still
match in the {{DelegationTokenSelector}} case because {{buildTokenService}}
normalizes the brackets. The problem is that the two resolvers disagree on the
same input.
*Proposed fix:* in {{QualifiedHostResolver.resolve}}, remove one pair of
surrounding brackets before the {{isInetAddress}} check, so a bracketed literal
is treated as an IP address in both modes. Add tests for {{[::1]}} and
{{[2001:db8::1]}} with {{use_ip}} set to true and false.
POC (part of a larger change):
https://github.com/smengcl/hadoop/commit/784fc0c2e78017c3eaa3a8012cc2d0a1a6e53d6b
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
