zhangshuyan0 opened a new pull request, #5941: URL: https://github.com/apache/hadoop/pull/5941
In the method `updateBlockForPipeline`, NameNode uses the `BlockUnderConstructionFeature` of a BlockInfo to generate the member `blockIndices` of `LocatedStripedBlock`. https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L5308-L5319 And then, it uses `blockIndices` to generate block tokens for client. https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L1618-L1632 However, if there is a failover, the location info in BlockUnderConstructionFeature may be incomplete, which results in the absence of the corresponding block tokens. When the client receives these incomplete block tokens, it will throw a NPE because `updatedBlks[i]` is null (line 825). https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java#L820-L828 As a result, the write process in client fails. We need to fix this bug. NameNode should just return block tokens for all indices to the client. Client can pick whichever it likes to use. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
