Huaxiang Sun created HDFS-16540: ----------------------------------- Summary: Data locality is lost when DataNode pod restarts in kubernetes Key: HDFS-16540 URL: https://issues.apache.org/jira/browse/HDFS-16540 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.3.2 Reporter: Huaxiang Sun
We have HBase RegionServer and Hdfs DataNode running in one pod. When the pod restarts, we found that data locality is lost after we do a major compaction of hbase regions. After some debugging, we found that upon pod restarts, its ip changes. In DatanodeManager, maps like networktopology are updated with the new info. host2DatanodeMap is not updated accordingly. When hdfs client with the new ip tries to find a local DataNode, it fails. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org