ChenSammi commented on PR #6331:
URL: https://github.com/apache/ozone/pull/6331#issuecomment-1982879868
> This can cause the payload in the RocksDB to increase by 64x. I understand
the need to avoid client to not do wasteful work. Should we benchmark a single
stream performance for 16, 32, 64, 128 kb?
@kerneltime , thanks for the review. The RocksDB data size change is a valid
point. After a further investigation, here is the data. Currently following
data structures are used to save block data in RocksDB
```
message BlockData {
required DatanodeBlockID blockID = 1;
optional int64 flags = 2; // for future use.
repeated KeyValue metadata = 3;
repeated ChunkInfo chunks = 4;
optional int64 size = 5;
}
message ChunkInfo {
required string chunkName = 1;
required uint64 offset = 2;
required uint64 len = 3;
repeated KeyValue metadata = 4;
required ChecksumData checksumData =5;
optional bytes stripeChecksum = 6;
}
message ChecksumData {
required ChecksumType type = 1;
required uint32 bytesPerChecksum = 2;
repeated bytes checksums = 3;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
