[
https://issues.apache.org/jira/browse/HADOOP-19961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18126643#comment-18126643
]
ASF GitHub Bot commented on HADOOP-19961:
-----------------------------------------
smengcl opened a new pull request, #8819:
URL: https://github.com/apache/hadoop/pull/8819
Generated-by: Claude Code (Opus 5.5)
### Description of PR
`KMSClientProvider.Factory` splits the authority of a `kms://` provider URI
on every colon, so a KMS cannot be addressed by an IPv6 literal.
`kms://https@[2001:db8::1]:9600/kms` in `hadoop.security.key.provider.path`
fails with `Could not parse port in kms uri`. Every caller creates the provider
through this factory (`KMSUtil.createKeyProviderFromUri`, used by the HDFS
client, the NameNode and the token renewer), so there is no working way to
configure such a KMS today. DNS names and IPv4 addresses are not affected.
This PR adds a branch to the factory for an authority that contains a
bracketed IPv6 literal.
* A single host is written with literal brackets:
`kms://https@[2001:db8::1]:9600/kms`.
* In a host list the brackets must be percent-encoded:
`kms://https@%5B2001:db8::1%5D;host2;10.0.0.3:9600/kms`. `java.net.URI` rejects
literal brackets next to a `;`, and the provider path, the URI the NameNode
hands to clients and the token service are all parsed with `URI.create`. The
percent-encoded form is the only spelling that `URI` accepts, and
`URI.getAuthority()` decodes it to the existing `host1;host2:port` form.
* All hosts share one port, as before. The port is what follows a colon
after the last closing bracket.
* IPv6, IPv4 and DNS hosts can be mixed in one list. The URL of each host
keeps the brackets, for example `https://[2001:db8::1]:9600/kms/v1/`.
* An IPv6 literal without brackets is rejected, because the end of the
address and the port cannot be told apart. Malformed brackets, a port per host
and an empty host are rejected with `IOException`.
* The token service is the provider URI as written, so `KMSTokenRenewer` can
parse it again. For a provider URI with a fragment, `getDtService` now keeps
the percent-encoded brackets, since the decoded form would not parse.
An authority without a bracket takes the existing code path, so every
provider URI that works on trunk yields the same provider URLs and token
service. A provider URI without a port still fails, as it does on trunk for a
DNS name, because the token service needs a port.
The KMS documentation gets a short note on the syntax.
Sub-task of HADOOP-11890.
### How was this patch tested?
Added 4 tests to `TestLoadBalancingKMSClientProvider`, which assert on the
provider URLs created through `KMSUtil.createKeyProviderFromUri`:
* `testCreationWithIPv6`: a single bracketed host, a list of IPv6 hosts,
lists that mix DNS, IPv4 and IPv6 hosts with the IPv6 host in each position,
and a DNS and IPv4 list to show that existing behavior is kept.
* `testCreationWithoutPort`: an IPv6 host without a port fails in the same
way as a DNS name without a port.
* `testCreationWithMalformedIPv6`: unbracketed literals in each position,
unclosed brackets, brackets around a name, trailing characters, a port per
host, a port that is not a number, an empty host and a missing protocol are
rejected.
* `testTokenServiceWithIPv6`: the token service equals the provider URI
(also when the URI has a fragment), creates the same providers when parsed
again, and is the bracketed address of the first host when the provider path is
configured.
On trunk all 4 new tests fail.
`mvn test -Dtest='TestKMSClientProvider,TestLoadBalancingKMSClientProvider'`
in `hadoop-common` with JDK 17: 34 tests run, 0 failures, 0 errors, 0 skipped.
Also checked manually that a provider created from
`kms://http@[::1]:<port>/kms`, and one created from a percent-encoded list,
send their requests to a local HTTP server bound to `::1` with the expected
path and `Host` header.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the
endpoint
declared according to the connector-specific documentation? *Note:
Automated CI
testing doesn't cover all cases so manual testing with cloud storage
is still
required.*
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
### AI Tooling
Contains content generated by Claude Code.
- [x] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
🤖 Generated with [Claude Code](https://claude.com/claude-code)
> Support IPv6 authorities in KMS client provider URIs
> ----------------------------------------------------
>
> Key: HADOOP-19961
> URL: https://issues.apache.org/jira/browse/HADOOP-19961
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> KMSClientProvider splits authorities on a colon. This parsing fails for IPv6
> literals. Support bracketed single-host authorities and percent-encoded
> multi-host failover authorities. Use one shared port. Reject ambiguous
> unbracketed IPv6 literals. Preserve DNS and IPv4 behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
