[ https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17753114#comment-17753114 ]
ASF GitHub Bot commented on HDFS-17154: --------------------------------------- zhangshuyan0 opened a new pull request, #5941: URL: https://github.com/apache/hadoop/pull/5941 In the method `updateBlockForPipeline`, NameNode uses the `BlockUnderConstructionFeature` of a BlockInfo to generate the member `blockIndices` of `LocatedStripedBlock`. https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L5308-L5319 And then, it uses `blockIndices` to generate block tokens for client. https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L1618-L1632 However, if there is a failover, the location info in BlockUnderConstructionFeature may be incomplete, which results in the absence of the corresponding block tokens. When the client receives these incomplete block tokens, it will throw a NPE because `updatedBlks[i]` is null (line 825). https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java#L820-L828 As a result, the write process in client fails. We need to fix this bug. NameNode should just return block tokens for all indices to the client. Client can pick whichever it likes to use. > EC: Fix bug in updateBlockForPipeline after failover > ---------------------------------------------------- > > Key: HDFS-17154 > URL: https://issues.apache.org/jira/browse/HDFS-17154 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Shuyan Zhang > Assignee: Shuyan Zhang > Priority: Major > > In the method `updateBlockForPipeline`, NameNode uses the > `BlockUnderConstructionFeature` of a BlockInfo to generate the member > `blockIndices` of `LocatedStripedBlock`. > And then, NameNode uses `blockIndices` to generate block tokens for client. > However, if there is a failover, the location info in > BlockUnderConstructionFeature may be incomplete, which results in the absence > of the corresponding block tokens. > When the client receives these incomplete block tokens, it will throw a NPE > because `updatedBlks[i]` is null. > NameNode should just return block tokens for all indices to the client. > Client can pick whichever it likes to use. > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org