[
https://issues.apache.org/jira/browse/HDDS-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698855#comment-17698855
]
Ivan Andika commented on HDDS-8131:
-----------------------------------
Thank you very much for the feedback [~Nibiruxu].
For 1. Currently our cluster still uses Ratis 2.2.0 which does not have this
preservation feature yet so I can't rely on it for now. We're planning to
upgrade to 2.4.1 soon.
For 2. Do you mean to turn it off? In Ozone it's turned on by default, which is
why the logs are purged up to the {{snapshotIndex}}
For 3. From what I understand {{raft.server.log.purge.gap}} is not a reliable
way to ensure that the follower NEVER download snapshot from the leader (since
it's too large). There is still a chance where the log will be purged up to
the {{{}snapshotIndex{}}}. {{suggestedIndex}} can either be 1.
{{snapshotIndex}} (if option 2 is enabled) or 2. lowest of all the peers
{{commitIndex}} (if option 2 is disabled). So if for some reason
{{{}suggestedIndex - lastPurge > purgeGap{}}}, the logs are still purged up to
the {{snapshotIndex}} without checking whether the slow follower has replicated
it in its log.
Regarding HDDS-6961, I will let the reviewer know.
> Add Configuration for OM Ratis Log Purge Tuning Parameters
> ----------------------------------------------------------
>
> Key: HDDS-8131
> URL: https://issues.apache.org/jira/browse/HDDS-8131
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0
>
>
> Currently Ozone Manager enables {{raft.server.log.purge.upto.snapshot.index}}
> by default.
> However, for OM cluster with large metadata store, there might be a case
> where OM leader purge its Ratis logs before a slow follower replicated it to
> its log. This means that the follower needs to download the whole metadata
> store from the OM leader. This can be problematic if the metadata store in
> leader is too large.
> We should add two configurations in OM to enable/disable Ratis purge
> parameters:
> * {{raft.server.log.purge.upto.snapshot.index}}
> ** Disabling this would guarantee that the OM leader will not purge its
> Ratis log unless all the logs have been replicated to all the followers
> (through {{{}commitIndex{}}}).
> ** This would effectively means that there shouldn't be a case where the
> slow follower needs to download the full metadata from the leader. So no
> snapshot download from follower. For small OM metadata, it can be faster for
> follower to download the leader's metadata snapshot than normally replicating
> and applying the outstanding logs.
> ** For a very slow follower / downed follower, the OM leader cannot purge
> the log until the follower catch up to it. This might increase the disk space
> usage for OM leader.
> ** Default would be {{true}} to preserve the current OM snapshot behavior
> * {{raft.server.log.purge.preservation.log.num}}
> ** RATIS-1626 introduces logic to preserve the latest n won't-be-purged logs
> ** Setting n > 0 while still enabling
> {{raft.server.log.purge.upto.snapshot.index}} should balance a between the
> cost of preserving & transferring logs and the cost of transferring snapshot.
> ** Default would be 0 to preserve the current OM snapshot behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
