[
https://issues.apache.org/jira/browse/HDDS-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598141#comment-17598141
]
Kaijie Chen commented on HDDS-7177:
-----------------------------------
{quote}
This can happen only when first stripe failed. It is ok when non first stripe
failed right.
I agree, in general we need to bring the retry logic though. However this
problem will not be fixed with that fully.
{quote}
If the first stripe failed, the block file is not tracked in metadata.
If some later stripe failed, the block file will be 1 chunk larger than what's
in metadata.
And if rewrite happens often, the block files on disk will be small and in
different sizes.
The following example shows writing some 1000M keys in EC/10-4-1024k,
the ideal block size will be all in 100M. But due to rewrite, it ends up like
this:
(there is no node failure in this cluster, it is caused by network congestion)
{noformat}
68 1.0M 116 2.0M 81 3.0M 61 4.0M 91 5.0M 112 6.0M 69 7.0M 91 8.0M 76 9.0M 64
10M 47 11M 56 12M 72 13M 75 14M 71 15M 90 16M 57 17M 121 18M 82 19M 72 20M 48
21M 61 22
M 88 23M 95 24M 80 25M 84 26M 78 27M 96 28M 45 29M 74 30M 39 31M 39 32M 81 33M
58 34M 57 35M 22 36M 85 37M 91 38M 50 39M 32 40M 54 41M 51 42M 71 43M 57 44M 72
45M 85 46M 47 47M 59 48M 60 49M 78 5
0M 92 51M 68 52M 54 53M 39 54M 93 55M 46 56M 36 57M 48 58M 38 59M 29 60M 42 61M
26 62M 34 63M 77 64M 16 65M 57 66M 30 67M 33 68M 16 69M 20 70M 34 71M 17 72M 44
73M 29 74M 30 75M 34 76M 38 77M 22
78M 20 79M 31 80M 30 81M 32 82M 28 83M 10 84M 32 85M 13 86M 21 87M 25 88M 13
89M 31 90M 11 91M 27 92M 30 93M 25 94M 53 95M 21 96M 16 97M 9 98M 27 99M 1445
100M
{noformat}
> EC produces some unknown 1MB blocks without the control of deleting service
> ---------------------------------------------------------------------------
>
> Key: HDDS-7177
> URL: https://issues.apache.org/jira/browse/HDDS-7177
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Xu Shao Hong
> Assignee: Xu Shao Hong
> Priority: Major
> Attachments: image-2022-08-26-16-28-26-251.png
>
>
> We found that even deleted all the EC keys in the cluster and flushed the
> transactionbuffer, there are still some blocks left. They are all of 1M size.
> The config is rs-10-4-1024k
> This could be replayed.
> !image-2022-08-26-16-28-26-251.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
