[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[ https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755032#comment-17755032 ] ASF GitHub Bot commented on HDFS-17154: --- Hexiaoqiao commented on PR #5941: URL: https://github.com/apache/hadoop/pull/5941#issuecomment-1680515885 Committed to trunk. Thanks @zhangshuyan0 for your works and @haiyang1987 for your reviews! Note: The failed unit test is not related to this changes. > EC: Fix bug in updateBlockForPipeline after failover > > > Key: HDFS-17154 > URL: https://issues.apache.org/jira/browse/HDFS-17154 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shuyan Zhang >Assignee: Shuyan Zhang >Priority: Major > Labels: pull-request-available > > In the method `updateBlockForPipeline`, NameNode uses the > `BlockUnderConstructionFeature` of a BlockInfo to generate the member > `blockIndices` of `LocatedStripedBlock`. > And then, NameNode uses `blockIndices` to generate block tokens for client. > However, if there is a failover, the location info in > BlockUnderConstructionFeature may be incomplete, which results in the absence > of the corresponding block tokens. > When the client receives these incomplete block tokens, it will throw a NPE > because `updatedBlks[i]` is null. > NameNode should just return block tokens for all indices to the client. > Client can pick whichever it likes to use. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[ https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755031#comment-17755031 ] ASF GitHub Bot commented on HDFS-17154: --- Hexiaoqiao merged PR #5941: URL: https://github.com/apache/hadoop/pull/5941 > EC: Fix bug in updateBlockForPipeline after failover > > > Key: HDFS-17154 > URL: https://issues.apache.org/jira/browse/HDFS-17154 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shuyan Zhang >Assignee: Shuyan Zhang >Priority: Major > Labels: pull-request-available > > In the method `updateBlockForPipeline`, NameNode uses the > `BlockUnderConstructionFeature` of a BlockInfo to generate the member > `blockIndices` of `LocatedStripedBlock`. > And then, NameNode uses `blockIndices` to generate block tokens for client. > However, if there is a failover, the location info in > BlockUnderConstructionFeature may be incomplete, which results in the absence > of the corresponding block tokens. > When the client receives these incomplete block tokens, it will throw a NPE > because `updatedBlks[i]` is null. > NameNode should just return block tokens for all indices to the client. > Client can pick whichever it likes to use. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[ https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755015#comment-17755015 ] ASF GitHub Bot commented on HDFS-17154: --- haiyang1987 commented on PR #5941: URL: https://github.com/apache/hadoop/pull/5941#issuecomment-1680411872 LGTM. +1 > EC: Fix bug in updateBlockForPipeline after failover > > > Key: HDFS-17154 > URL: https://issues.apache.org/jira/browse/HDFS-17154 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shuyan Zhang >Assignee: Shuyan Zhang >Priority: Major > Labels: pull-request-available > > In the method `updateBlockForPipeline`, NameNode uses the > `BlockUnderConstructionFeature` of a BlockInfo to generate the member > `blockIndices` of `LocatedStripedBlock`. > And then, NameNode uses `blockIndices` to generate block tokens for client. > However, if there is a failover, the location info in > BlockUnderConstructionFeature may be incomplete, which results in the absence > of the corresponding block tokens. > When the client receives these incomplete block tokens, it will throw a NPE > because `updatedBlks[i]` is null. > NameNode should just return block tokens for all indices to the client. > Client can pick whichever it likes to use. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[
https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754384#comment-17754384
]
ASF GitHub Bot commented on HDFS-17154:
---
Hexiaoqiao commented on code in PR #5941:
URL: https://github.com/apache/hadoop/pull/5941#discussion_r1294147667
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##
@@ -5950,8 +5952,26 @@ LocatedBlock bumpBlockGenerationStamp(ExtendedBlock
block,
block.setGenerationStamp(nextGenerationStamp(
blockManager.isLegacyBlock(block.getLocalBlock(;
- locatedBlock = BlockManager.newLocatedBlock(
- block, file.getLastBlock(), null, -1);
+ BlockInfo lastBlockInfo = file.getLastBlock();
+ locatedBlock = BlockManager.newLocatedBlock(block, lastBlockInfo,
+ null, -1);
+ if (lastBlockInfo.isStriped() &&
+ ((BlockInfoStriped) lastBlockInfo).getTotalBlockNum() >
+ ((LocatedStripedBlock) locatedBlock).getBlockIndices().length) {
+// The location info in BlockUnderConstructionFeature may not be
+// complete after a failover, so we just return all block tokens for a
+// striped block. This will disrupt the correspondence between
+// LocatedStripedBlock.blockIndices and LocatedStripedBlock.locs,
+// which is not used in client side. The correspondence between
+// LocatedStripedBlock.blockIndices and LocatedBlock.blockToken is
+// ensured.
+byte[] indices =
+new byte[((BlockInfoStriped) lastBlockInfo).getTotalBlockNum()];
+for (int i = 0; i < indices.length; ++i) {
+ indices[i] = (byte) i;
Review Comment:
Got it. +1.
> EC: Fix bug in updateBlockForPipeline after failover
>
>
> Key: HDFS-17154
> URL: https://issues.apache.org/jira/browse/HDFS-17154
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Shuyan Zhang
>Assignee: Shuyan Zhang
>Priority: Major
> Labels: pull-request-available
>
> In the method `updateBlockForPipeline`, NameNode uses the
> `BlockUnderConstructionFeature` of a BlockInfo to generate the member
> `blockIndices` of `LocatedStripedBlock`.
> And then, NameNode uses `blockIndices` to generate block tokens for client.
> However, if there is a failover, the location info in
> BlockUnderConstructionFeature may be incomplete, which results in the absence
> of the corresponding block tokens.
> When the client receives these incomplete block tokens, it will throw a NPE
> because `updatedBlks[i]` is null.
> NameNode should just return block tokens for all indices to the client.
> Client can pick whichever it likes to use.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[
https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754373#comment-17754373
]
ASF GitHub Bot commented on HDFS-17154:
---
zhangshuyan0 commented on code in PR #5941:
URL: https://github.com/apache/hadoop/pull/5941#discussion_r1294132630
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##
@@ -5950,8 +5952,26 @@ LocatedBlock bumpBlockGenerationStamp(ExtendedBlock
block,
block.setGenerationStamp(nextGenerationStamp(
blockManager.isLegacyBlock(block.getLocalBlock(;
- locatedBlock = BlockManager.newLocatedBlock(
- block, file.getLastBlock(), null, -1);
+ BlockInfo lastBlockInfo = file.getLastBlock();
+ locatedBlock = BlockManager.newLocatedBlock(block, lastBlockInfo,
+ null, -1);
+ if (lastBlockInfo.isStriped() &&
+ ((BlockInfoStriped) lastBlockInfo).getTotalBlockNum() >
+ ((LocatedStripedBlock) locatedBlock).getBlockIndices().length) {
+// The location info in BlockUnderConstructionFeature may not be
+// complete after a failover, so we just return all block tokens for a
+// striped block. This will disrupt the correspondence between
+// LocatedStripedBlock.blockIndices and LocatedStripedBlock.locs,
+// which is not used in client side. The correspondence between
+// LocatedStripedBlock.blockIndices and LocatedBlock.blockToken is
+// ensured.
+byte[] indices =
+new byte[((BlockInfoStriped) lastBlockInfo).getTotalBlockNum()];
+for (int i = 0; i < indices.length; ++i) {
+ indices[i] = (byte) i;
Review Comment:
The role of `byte[] indices` here is to generate block tokens. In your
example, if the client does not need [1,6,7], it will directly skip the
corresponding block tokens and not use them.
> EC: Fix bug in updateBlockForPipeline after failover
>
>
> Key: HDFS-17154
> URL: https://issues.apache.org/jira/browse/HDFS-17154
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Shuyan Zhang
>Assignee: Shuyan Zhang
>Priority: Major
> Labels: pull-request-available
>
> In the method `updateBlockForPipeline`, NameNode uses the
> `BlockUnderConstructionFeature` of a BlockInfo to generate the member
> `blockIndices` of `LocatedStripedBlock`.
> And then, NameNode uses `blockIndices` to generate block tokens for client.
> However, if there is a failover, the location info in
> BlockUnderConstructionFeature may be incomplete, which results in the absence
> of the corresponding block tokens.
> When the client receives these incomplete block tokens, it will throw a NPE
> because `updatedBlks[i]` is null.
> NameNode should just return block tokens for all indices to the client.
> Client can pick whichever it likes to use.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[
https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753927#comment-17753927
]
ASF GitHub Bot commented on HDFS-17154:
---
Hexiaoqiao commented on code in PR #5941:
URL: https://github.com/apache/hadoop/pull/5941#discussion_r1293059482
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##
@@ -5950,8 +5952,26 @@ LocatedBlock bumpBlockGenerationStamp(ExtendedBlock
block,
block.setGenerationStamp(nextGenerationStamp(
blockManager.isLegacyBlock(block.getLocalBlock(;
- locatedBlock = BlockManager.newLocatedBlock(
- block, file.getLastBlock(), null, -1);
+ BlockInfo lastBlockInfo = file.getLastBlock();
+ locatedBlock = BlockManager.newLocatedBlock(block, lastBlockInfo,
+ null, -1);
+ if (lastBlockInfo.isStriped() &&
+ ((BlockInfoStriped) lastBlockInfo).getTotalBlockNum() >
+ ((LocatedStripedBlock) locatedBlock).getBlockIndices().length) {
+// The location info in BlockUnderConstructionFeature may not be
+// complete after a failover, so we just return all block tokens for a
+// striped block. This will disrupt the correspondence between
+// LocatedStripedBlock.blockIndices and LocatedStripedBlock.locs,
+// which is not used in client side. The correspondence between
+// LocatedStripedBlock.blockIndices and LocatedBlock.blockToken is
+// ensured.
+byte[] indices =
+new byte[((BlockInfoStriped) lastBlockInfo).getTotalBlockNum()];
+for (int i = 0; i < indices.length; ++i) {
+ indices[i] = (byte) i;
Review Comment:
Was the index continuous here? is it possible that the seq is [0,2,3,4,5,8]
for one 6-3-1024k files?
> EC: Fix bug in updateBlockForPipeline after failover
>
>
> Key: HDFS-17154
> URL: https://issues.apache.org/jira/browse/HDFS-17154
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Shuyan Zhang
>Assignee: Shuyan Zhang
>Priority: Major
> Labels: pull-request-available
>
> In the method `updateBlockForPipeline`, NameNode uses the
> `BlockUnderConstructionFeature` of a BlockInfo to generate the member
> `blockIndices` of `LocatedStripedBlock`.
> And then, NameNode uses `blockIndices` to generate block tokens for client.
> However, if there is a failover, the location info in
> BlockUnderConstructionFeature may be incomplete, which results in the absence
> of the corresponding block tokens.
> When the client receives these incomplete block tokens, it will throw a NPE
> because `updatedBlks[i]` is null.
> NameNode should just return block tokens for all indices to the client.
> Client can pick whichever it likes to use.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[
https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753264#comment-17753264
]
ASF GitHub Bot commented on HDFS-17154:
---
hadoop-yetus commented on PR #5941:
URL: https://github.com/apache/hadoop/pull/5941#issuecomment-1675009020
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 0s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 53s | | trunk passed |
| +1 :green_heart: | compile | 5m 22s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 5m 18s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 55s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 18s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 5m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 44s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 54s | | the patch passed |
| +1 :green_heart: | compile | 5m 17s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 5m 17s | | the patch passed |
| +1 :green_heart: | compile | 5m 6s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 5m 6s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 5m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 24s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 217m 13s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5941/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 386m 2s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5941/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5941 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 63c5ebac5f2b 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 19b4726482db20b9d7a9351ce517b298755d71a0 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
[jira] [Commented] (HDFS-17154) EC: Fix bug in updateBlockForPipeline after failover
[ https://issues.apache.org/jira/browse/HDFS-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753114#comment-17753114 ] ASF GitHub Bot commented on HDFS-17154: --- zhangshuyan0 opened a new pull request, #5941: URL: https://github.com/apache/hadoop/pull/5941 In the method `updateBlockForPipeline`, NameNode uses the `BlockUnderConstructionFeature` of a BlockInfo to generate the member `blockIndices` of `LocatedStripedBlock`. https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L5308-L5319 And then, it uses `blockIndices` to generate block tokens for client. https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L1618-L1632 However, if there is a failover, the location info in BlockUnderConstructionFeature may be incomplete, which results in the absence of the corresponding block tokens. When the client receives these incomplete block tokens, it will throw a NPE because `updatedBlks[i]` is null (line 825). https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java#L820-L828 As a result, the write process in client fails. We need to fix this bug. NameNode should just return block tokens for all indices to the client. Client can pick whichever it likes to use. > EC: Fix bug in updateBlockForPipeline after failover > > > Key: HDFS-17154 > URL: https://issues.apache.org/jira/browse/HDFS-17154 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shuyan Zhang >Assignee: Shuyan Zhang >Priority: Major > > In the method `updateBlockForPipeline`, NameNode uses the > `BlockUnderConstructionFeature` of a BlockInfo to generate the member > `blockIndices` of `LocatedStripedBlock`. > And then, NameNode uses `blockIndices` to generate block tokens for client. > However, if there is a failover, the location info in > BlockUnderConstructionFeature may be incomplete, which results in the absence > of the corresponding block tokens. > When the client receives these incomplete block tokens, it will throw a NPE > because `updatedBlks[i]` is null. > NameNode should just return block tokens for all indices to the client. > Client can pick whichever it likes to use. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
