Alexey Serbin created KUDU-3453:
-----------------------------------
Summary: Fine-grained anchoring for WAL segments during tablet
copying
Key: KUDU-3453
URL: https://issues.apache.org/jira/browse/KUDU-3453
Project: Kudu
Issue Type: Improvement
Components: tablet, tserver
Reporter: Alexey Serbin
Tablet copying is a provision to implement the process of automatic tablet
re-replication in Kudu. When the system catalog (Kudu master) detects that a
tablet replica is no longer available, it automatically re-replicates a tablet
to a destination tablet server using another healthy tablet replica in the
cluster as the source.
When copying a tablet from one tablet server to another, the source tablet
copying session "anchors" WAL segments to be transfered to the destination
server, so they are not GC-ed by the tablet maintenance operation when they are
no longer needed locally, but the tablet copy session is still in progress.
The anchored WAL segments are releases all at once when the tablet copying
session completes with success of failure. However, there might be long
running tablet copying sessions, and with high data ingest rate, the source
tablet replica might accumulate huge amount of WAL data which isn't relevant at
both the source and the destination server.
To prevent accumulation of WAL data for long-running tablet copying sessions,
it's necessary to update the WAL anchors in a more granular manner, e.g.
un-anchor a segment once it has been successfully copied and persisted by the
client tablet copying session.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
