[
https://issues.apache.org/jira/browse/KUDU-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Serbin updated KUDU-3453:
--------------------------------
Summary: Fine-grained anchoring for WAL segments for tablet copy (was:
Fine-grained anchoring for WAL segments during tablet copying)
> Fine-grained anchoring for WAL segments for tablet copy
> -------------------------------------------------------
>
> Key: KUDU-3453
> URL: https://issues.apache.org/jira/browse/KUDU-3453
> Project: Kudu
> Issue Type: Improvement
> Components: tablet, tserver
> Reporter: Alexey Serbin
> Priority: Major
>
> Tablet copying is a provision to implement the process of automatic tablet
> re-replication in Kudu. When the system catalog (Kudu master) detects that a
> tablet replica is no longer available, it automatically re-replicates a
> tablet to a destination tablet server using another healthy tablet replica in
> the cluster as the source.
> When copying a tablet from one tablet server to another, the source tablet
> copying session "anchors" WAL segments to be transfered to the destination
> server, so they are not GC-ed by the tablet maintenance operation when they
> are no longer needed locally, but the tablet copy session is still in
> progress.
> The anchored WAL segments are releases all at once when the tablet copying
> session completes with success of failure. However, there might be long
> running tablet copying sessions, and with high data ingest rate, the source
> tablet replica might accumulate huge amount of WAL data which isn't relevant
> at both the source and the destination server.
> To prevent accumulation of WAL data for long-running tablet copying sessions,
> it's necessary to update the WAL anchors in a more granular manner, e.g.
> un-anchor a segment once it has been successfully copied and persisted by the
> client tablet copying session.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
