smengcl opened a new pull request, #8819: URL: https://github.com/apache/hadoop/pull/8819
Generated-by: Claude Code (Opus 5.5) ### Description of PR `KMSClientProvider.Factory` splits the authority of a `kms://` provider URI on every colon, so a KMS cannot be addressed by an IPv6 literal. `kms://https@[2001:db8::1]:9600/kms` in `hadoop.security.key.provider.path` fails with `Could not parse port in kms uri`. Every caller creates the provider through this factory (`KMSUtil.createKeyProviderFromUri`, used by the HDFS client, the NameNode and the token renewer), so there is no working way to configure such a KMS today. DNS names and IPv4 addresses are not affected. This PR adds a branch to the factory for an authority that contains a bracketed IPv6 literal. * A single host is written with literal brackets: `kms://https@[2001:db8::1]:9600/kms`. * In a host list the brackets must be percent-encoded: `kms://https@%5B2001:db8::1%5D;host2;10.0.0.3:9600/kms`. `java.net.URI` rejects literal brackets next to a `;`, and the provider path, the URI the NameNode hands to clients and the token service are all parsed with `URI.create`. The percent-encoded form is the only spelling that `URI` accepts, and `URI.getAuthority()` decodes it to the existing `host1;host2:port` form. * All hosts share one port, as before. The port is what follows a colon after the last closing bracket. * IPv6, IPv4 and DNS hosts can be mixed in one list. The URL of each host keeps the brackets, for example `https://[2001:db8::1]:9600/kms/v1/`. * An IPv6 literal without brackets is rejected, because the end of the address and the port cannot be told apart. Malformed brackets, a port per host and an empty host are rejected with `IOException`. * The token service is the provider URI as written, so `KMSTokenRenewer` can parse it again. For a provider URI with a fragment, `getDtService` now keeps the percent-encoded brackets, since the decoded form would not parse. An authority without a bracket takes the existing code path, so every provider URI that works on trunk yields the same provider URLs and token service. A provider URI without a port still fails, as it does on trunk for a DNS name, because the token service needs a port. The KMS documentation gets a short note on the syntax. Sub-task of HADOOP-11890. ### How was this patch tested? Added 4 tests to `TestLoadBalancingKMSClientProvider`, which assert on the provider URLs created through `KMSUtil.createKeyProviderFromUri`: * `testCreationWithIPv6`: a single bracketed host, a list of IPv6 hosts, lists that mix DNS, IPv4 and IPv6 hosts with the IPv6 host in each position, and a DNS and IPv4 list to show that existing behavior is kept. * `testCreationWithoutPort`: an IPv6 host without a port fails in the same way as a DNS name without a port. * `testCreationWithMalformedIPv6`: unbracketed literals in each position, unclosed brackets, brackets around a name, trailing characters, a port per host, a port that is not a number, an empty host and a missing protocol are rejected. * `testTokenServiceWithIPv6`: the token service equals the provider URI (also when the URI has a fragment), creates the same providers when parsed again, and is the bracketed address of the first host when the provider path is configured. On trunk all 4 new tests fail. `mvn test -Dtest='TestKMSClientProvider,TestLoadBalancingKMSClientProvider'` in `hadoop-common` with JDK 17: 34 tests run, 0 failures, 0 errors, 0 skipped. Also checked manually that a provider created from `kms://http@[::1]:<port>/kms`, and one created from a percent-encoded list, send their requests to a local HTTP server bound to `::1` with the expected path and `Host` header. ### For code changes: - [x] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: Have the integration tests been executed and the endpoint declared according to the connector-specific documentation? *Note: Automated CI testing doesn't cover all cases so manual testing with cloud storage is still required.* - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? ### AI Tooling Contains content generated by Claude Code. - [x] The PR includes the phrase "Contains content generated by <tool>" where <tool> is the name of the AI tool used. - [x] My use of AI contributions follows the ASF legal policy https://www.apache.org/legal/generative-tooling.html 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
