smengcl opened a new pull request, #8958: URL: https://github.com/apache/hadoop/pull/8958
Generated-by: Claude Code (Opus 5.5) JIRA: https://issues.apache.org/jira/browse/HDFS-18012 ### Description of PR `NNStorage.newBlockPoolID` puts the NameNode's default IP into the block pool ID. With an IPv6 address the ID contains colons (and possibly a `%scope`), for example `BP-1234-fd00:dead:beef:0:0:0:0:10-1794005284284`. The ID is used as a directory name on every DataNode volume, and DataNode storage initialization fails in `StorageLocation.makeBlockPoolDir`, where `ChecksumFileSystem` builds a `.crc` path from the directory name: ``` java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .BP-173958434-fd00:dead:beef:0:0:0:0:10-1794005284284.crc at org.apache.hadoop.fs.ChecksumFileSystem.getChecksumFile(ChecksumFileSystem.java:132) at org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:853) at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:116) at org.apache.hadoop.hdfs.server.datanode.StorageLocation.makeBlockPoolDir(StorageLocation.java:276) ``` So the DataNode never registers with such a NameNode. This change: - `NNStorage.newBlockPoolID`: drops the IPv6 scope ID and replaces `:` with `_`, giving `BP-1234-fd00_dead_beef_0_0_0_0_10-1794005284284`. IPv4 IDs are unchanged. - `BlockPoolSliceStorage`: the block pool ID regex (used for the DataNode storage root on upgrade, and for trash and restore during rolling upgrade) accepted only IPv4, so it now also accepts the IPv6 form. Existing block pool IDs are not touched, since the ID is generated only when a namespace is formatted. The general problem of colons in local file names under `ChecksumFileSystem` is HADOOP-17845. ### How was this patch tested? - `TestClusterId#testNewBlockPoolID` (new): IPv4 unchanged, IPv6 colons replaced, scope ID dropped. - `TestBlockPoolSliceStorage#testGetTrashAndRestoreDirectoriesWithIPv6` (new): fails without the regex change. - End to end, with a temporary local change forcing the NameNode IP to `fd00:dead:beef:0:0:0:0:10%eth0` (not part of this PR): on trunk `TestDataNodeRollingUpgrade` times out waiting for the cluster because of the exception above; with this patch all 6 tests pass with ID `BP-...-fd00_dead_beef_0_0_0_0_10-...`, including trash and restore. - Not run on a real IPv6 only host. ### For code changes: - [x] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: Have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? ### AI Tooling - [x] The PR includes the phrase "Contains content generated by Claude Code" where Claude Code is the name of the AI tool used. - [x] My use of AI contributions follows the ASF legal policy https://www.apache.org/legal/generative-tooling.html Contains content generated by Claude Code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
