sodonnel commented on PR #6613:
URL: https://github.com/apache/ozone/pull/6613#issuecomment-3324735795
The reason for the checksum error above, is this line in
`StreamBlockInputStream`:
```
if (verifyChecksum) {
ChecksumData checksumData = ChecksumData.getFromProtoBuf(
chunkInfo.getChecksumData());
int startIndex = (int) readChunk.getChunkData().getOffset() /
checksumData.getBytesPerChecksum();
```
The startIndex calculated needs to be hashed into the checksum position per
block. Eg, for a 16kb checksum, there are 256 checksums per 4MB chunk. So the
actual checksum index is `startIndex % 256`.
However, the problem is that we don't know the chunkSize in the client, and
different blocks could have different chunk sizes if the cluster setting was
changed at some point. Each readBlock response carries all the checksums for
that chunk and the block offset for the data it contains, but without knowing
the chunkSize I don't think there is a way to divide the overall block offset
down to the chunk level offset to index into the checksum array for that chunk.
ChunkInputStream works this out by comparing the requested offset with the
offset returned, its not clear to me why that works in the general case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
