[
https://issues.apache.org/jira/browse/HDDS-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17904640#comment-17904640
]
Hemant Kumar commented on HDDS-11745:
-------------------------------------
OM creates temporary dirs like *bootstrap-data-* in temp-bootstrap-data dir to
keep copy compaction-sst-backup dir a follower. On a happy path,
*bootstrap-data-* gets deleted after the response is returned to the client or
if there is a failure. But if JVM crashes, this dir is never deleted. We may
need to cleanup temp-bootstrap-data dir on OM restart. Since it depends on the
implementation of this task, keeping a note here.
> Use single tar to decouple filesystem snapshot layout from OM snapshot
> transfer
> -------------------------------------------------------------------------------
>
> Key: HDDS-11745
> URL: https://issues.apache.org/jira/browse/HDDS-11745
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ethan Rose
> Assignee: Hemant Kumar
> Priority: Major
>
> The current OM snapshot creation and transfer logic uses a list of SST files
> to track hardlink structure at the source and recreate it at the destination.
> It is also used to incrementally send the snapshots to followers by only
> sending a few SST files at a time. This adds extra complexity to the OM
> snapshot flow and prevents us from easily transferring and unpacking OM DB
> snapshots in other contexts like Recon sync or debug/repair.
> Tar preserves hardlinks on pack and unpack by default, so we can get the same
> space saving by packing all required files into a single tar, and then
> unpacking at the destination. The tar can be broken into pieces and streamed
> to followers to handle cases where it is large. This decouples network
> transfer from storage format.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
