[
https://issues.apache.org/jira/browse/HDDS-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850203#comment-17850203
]
Ethan Rose commented on HDDS-10379:
-----------------------------------
The current plan is to do this synchronously on container close: after the
close request goes through Ratis (for non-EC containers) but before acking the
close as successful back to SCM. This implementation will not hold up the
pipeline's state machine. The close container command handler currently uses
its own thread pool to handle commands, so adding a little overhead to these
commands should not affect other operations. It currently uses 3 threads to
handle the container close, but we can increase that number if needed.
> Datanode will generate Merkle Tree as part of container close
> -------------------------------------------------------------
>
> Key: HDDS-10379
> URL: https://issues.apache.org/jira/browse/HDDS-10379
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ritesh Shukla
> Priority: Major
>
> Open container replicas will not have a container merkle tree so that we do
> not need to keep updating this with every write request. Instead, the tree
> will be created on container close from the chunk checksums that are stored
> in RocksDB. Since these checksums should have been verified somewhat recently
> (with write checksum verification enabled), container close will build the
> merkle tree from these values without reading actual data blocks. Since
> should just be a hash of values stored contiguously in RocksDB, it should be
> a fast operation. The container scanner will come back asynchronously to
> update update the checksums based on the data it sees on disk at that time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
