[
https://issues.apache.org/jira/browse/HDDS-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nandakumar reassigned HDDS-7195:
--------------------------------
Assignee: Nandakumar
> EC: Additional checksum validations to make sure data integrity to avoid
> reconstruction time corruptions.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HDDS-7195
> URL: https://issues.apache.org/jira/browse/HDDS-7195
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Uma Maheswara Rao G
> Assignee: Nandakumar
> Priority: Major
>
> For EC blocks, each chunk has a checksum as for usual writes. The user data
> is written as a stripe across the container group, and each chunk has its own
> standalone checksum.
> After calculating parity and writing the parity bytes, we know the checksums
> for all chunks in the stripe, including parity. This is the stripe checksum,
> and it is a simple concatenation of all the checksum bytes for each chunk in
> the stripe in replicaIndex order. This stripe checksum is written to a
> redundant number of replicas in the container group - namely replica index =
> 1 and all parity replicas. In the case of EC-6-3, this means 4 out of the 9
> replicas hold the stripe checksum. For a container to be recovered at least 6
> replicas must be available, and hence the stripe checksum must be available
> if the container is recoverable.
> To ensure the integrity of each recovered chunk in a stripe, after recovering
> the data and calculating its checksum, we should use the stripe checksum
> already stored and compare the newly calculated checksum for the recovered
> chunk against that stored in the stripe checksum. If the new checksum does
> not match, the reconstruction should be failed. This will avoid the case
> where a chunk can get recovered with some error, due to a bug or wire
> corruption and it goes unnoticed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
