Jim Halfpenny created HDFS-17891:
------------------------------------
Summary: HostSet.add() throws uncaught exception if DataNode
hostname is not resolved
Key: HDFS-17891
URL: https://issues.apache.org/jira/browse/HDFS-17891
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 3.4.3
Reporter: Jim Halfpenny
The HostSet.add() method that adds DataNode addresses to a HostSet checks to
see if the hostname resolves. If this test fails then
Preconditions.checkArgument() throws an uncaught IllegalArgumentException:
{{ void add(InetSocketAddress addr) {}}
{{ Preconditions.checkArgument(!addr.isUnresolved());}}
{{ addrs.put(addr.getAddress(), addr.getPort());}}
It would make more sense log the failure and return from the add() method
rather than cause the creation of the HostSet to fail entirely. An example of
when this issue manifests is when running `hdfs dfsadmin -report` on a cluster
where one of the DataNode addresses cannot be resolved. The DataNode report
fails with an IllegalArgumentException:
{{stackable@simple-hdfs-namenode-default-0 /stackable/hadoop-3.4.0 $ hdfs
dfsadmin -report}}
{{Configured Capacity: 2933688373248 (2.67 TB)}}
{{Present Capacity: 1299099013120 (1.18 TB)}}
{{DFS Remaining: 1299098914816 (1.18 TB)}}
{{DFS Used: 98304 (96 KB)}}
{{DFS Used%: 0.00%}}
{{Replicated Blocks:}}
{{ Under replicated blocks: 0}}
{{ Blocks with corrupt replicas: 0}}
{{ Missing blocks: 0}}
{{ Missing blocks (with replication factor 1): 0}}
{{ Low redundancy blocks with highest priority to recover: 0}}
{{ Pending deletion blocks: 0}}
{{Erasure Coded Block Groups:}}
{{ Low redundancy block groups: 0}}
{{ Block groups with corrupt internal blocks: 0}}
{{ Missing block groups: 0}}
{{ Low redundancy blocks with highest priority to recover: 0}}
{{ Pending deletion blocks: 0}}
{{-------------------------------------------------}}
{{report: java.lang.IllegalArgumentException}}
The chance of a DataNode address not resolving is much more likely when HDFS is
being run in Kubernetes; if a DataNode pod is removed the corresponding DNS
entry will also be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
