[
https://issues.apache.org/jira/browse/HDDS-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-5359:
---------------------------------
Labels: pull-request-available (was: )
> Incorrect BLOCKCOUNT and BYTESUSED in container DB
> --------------------------------------------------
>
> Key: HDDS-5359
> URL: https://issues.apache.org/jira/browse/HDDS-5359
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Sammi Chen
> Assignee: Hanisha Koneru
> Priority: Major
> Labels: pull-request-available
> Attachments: negative.txt
>
>
> Here are default column family data of two different container replicas，
> "#BCSID" -> 1354765
> "#BLOCKCOUNT" -> -21
> "#BYTESUSED" -> 0
> "#PENDINGDELETEBLOCKCOUNT" -> 78
> "#delTX" -> 1141106
> "#BCSID" -> 1895040
> "#BLOCKCOUNT" -> -5
> "#BYTESUSED" -> 0
> "#PENDINGDELETEBLOCKCOUNT" -> 106
> "#delTX" -> 1146817
>
> Update:
> The BlockCount is incremented only when the Stream is closed and not when the
> BlockID is added to the DB. If the OutputStream was not closed properly or
> if, for any reason, the client starts writing to a new pipeline before the
> full block is written, it could lead to a Block being present in the
> container but the key_count (BlockCount) not being incremented for it. When a
> block is deleted from a container, the blockCount is also decremented. But if
> the blockCount is wrong to start with, it could lead to a negative value.
> When a block is deleted, usedBytes is decrement in memory first after
> deleting a chunk. And even if the chunkFile does not exist (already deleted),
> the usedBytes is decremented. This could lead to usedBytes being decremented
> multiple times for the same chunk and causing the total usedBytes metadata in
> the DB to become negative. Once all the chunks in all the blocks in that
> iteration of BlockDeletingService task are deleted, only then is the
> usedBytes updated in containerDB by taking the in-memory value. This Jira
> proposes to first update the DB with correct usedBytes (calculated from the
> BlockInfo after all chunks are deleted) and then update the in-memory
> metadata. This is the update sequence logic followed for all other state
> updates.
> Also, when a chunk is overwritten, then it is assumed that the size of the
> chunk remains the same. But it’s possible to overwrite more data into the
> chunk than originally present. In this case, the used_bytes should be updated
> with difference in the chunkSizes. (Adding this as a TODO).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
