[
https://issues.apache.org/jira/browse/HDFS-16833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628634#comment-17628634
]
ASF GitHub Bot commented on HDFS-16833:
---------------------------------------
tasanuma opened a new pull request, #5106:
URL: https://github.com/apache/hadoop/pull/5106
### Description of PR
JIRA: HDFS-16833
NameNode should log internal EC blocks instead of the EC block group when it
receives block reports.
Before this PR
```
// replica file
2022-11-04 10:38:20,124 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11007 is added to blk_1073741825_1001 (size=1024)
2022-11-04 10:38:20,126 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11004 is added to blk_1073741825_1001 (size=1024)
2022-11-04 10:38:20,126 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11001 is added to blk_1073741825_1001 (size=1024)
// ec file
2022-11-04 10:39:02,376 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11008 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,381 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11000 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,383 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11001 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,385 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11007 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,387 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11009 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,389 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11004 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,390 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11006 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,393 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11002 is added to blk_-9223372036854775792_1002 (size=0)
2022-11-04 10:39:02,395 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11003 is added to blk_-9223372036854775792_1002 (size=0)
```
After this PR
```
// replica file
2022-11-04 10:46:23,321 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11008 is added to blk_1073741825_1001 (size=1024)
2022-11-04 10:46:23,324 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11002 is added to blk_1073741825_1001 (size=1024)
2022-11-04 10:46:23,325 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11003 is added to blk_1073741825_1001 (size=1024)
// ec file
2022-11-04 10:47:53,469 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11009 is added to blk_-9223372036854775792_1002 (size=4194304)
2022-11-04 10:47:53,473 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11002 is added to blk_-9223372036854775791_1002 (size=4194304)
2022-11-04 10:47:53,474 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11006 is added to blk_-9223372036854775790_1002 (size=4194304)
2022-11-04 10:47:53,483 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11008 is added to blk_-9223372036854775789_1002 (size=4194304)
2022-11-04 10:47:53,485 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11003 is added to blk_-9223372036854775788_1002 (size=4194304)
2022-11-04 10:47:53,486 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11004 is added to blk_-9223372036854775787_1002 (size=4194304)
2022-11-04 10:47:53,488 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11001 is added to blk_-9223372036854775786_1002 (size=4194304)
2022-11-04 10:47:53,490 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11007 is added to blk_-9223372036854775785_1002 (size=4194304)
2022-11-04 10:47:53,491 [Block report processor] INFO BlockStateChange
(BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
127.0.0.1:11000 is added to blk_-9223372036854775784_1002 (size=4194304)
```
### How was this patch tested?
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
> NameNode should log internal EC blocks instead of the EC block group when it
> receives block reports
> ---------------------------------------------------------------------------------------------------
>
> Key: HDFS-16833
> URL: https://issues.apache.org/jira/browse/HDFS-16833
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: Takanobu Asanuma
> Assignee: Takanobu Asanuma
> Priority: Major
>
> When creating an EC file, NN only logs the EC block group for each of the
> internal EC block.
> {noformat}
> // replica file
> 2022-11-04 10:38:20,124 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11007 is added to blk_1073741825_1001 (size=1024)
> 2022-11-04 10:38:20,126 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11004 is added to blk_1073741825_1001 (size=1024)
> 2022-11-04 10:38:20,126 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11001 is added to blk_1073741825_1001 (size=1024)
> // ec file
> 2022-11-04 10:39:02,376 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11008 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,381 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11000 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,383 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11001 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,385 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11007 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,387 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11009 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,389 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11004 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,390 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11006 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,393 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11002 is added to blk_-9223372036854775792_1002 (size=0)
> 2022-11-04 10:39:02,395 [Block report processor] INFO BlockStateChange
> (BlockManager.java:addStoredBlock(3633)) - BLOCK* addStoredBlock:
> 127.0.0.1:11003 is added to blk_-9223372036854775792_1002 (size=0)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
