[
https://issues.apache.org/jira/browse/CASSANDRA-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-20920:
------------------------------------
Attachment: ci_summary-2.html
> CEP-45: Journal Replay
> -----------------------
>
> Key: CASSANDRA-20920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20920
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Consistency/Coordination
> Reporter: Alex Petrov
> Assignee: Alex Petrov
> Priority: High
> Attachments: ci_summary-1.html, ci_summary-2.html, ci_summary.html
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This patch introduces SegmentStateTracker, which tracks Mutation Journal
> allocations and listens to Memtable->SSTable flushes.
> * SegmentStateTracker logic is largely adapted from CommitLog (i.e.
> tracking dirty allocation upper bound, and clean/flushed min/max bounds, and
> checking their intersection to see if entire segment is flushed.
> * MutationJournal now tracks new allocations per segment (and, within a
> segment, per table), and listens to Memtable flushes, marking
> CommitLogPosition bounds reported by the flush as clean
> * Memtables now distinguish between commit log positions they track vs
> journal positions. It might be a good idea to mark CommitLogPosition with a
> corresponding flag to avoid accidentally passing a wrong position
> {{MutationJournal#replay}} is added, and serves a purpose similar to
> CommitLog replay, albeit lacks some of its functionality, such as replay
> filter, and DROP TABLE support for now (missing pieces documented inline).
> Only segments holding allocations that were not memtable->sstable flushed are
> considered for replay.
> Testing:
> * A fuzz test for SegmentStateTracker
> * A test that assumes some memtable behavior to exercise MutationJournal
> integration
> * Full integration / bounce test validating that data is being recovered by
> replay
> Important: a follow-up patch (in work by [~aleksey]) adds compaction and
> truncation of segments. Added tests will need to be expanded to support this.
> Remark: [trunk/accord commit I reviewed
> today|https://github.com/apache/cassandra/pull/4384/files#diff-a6dbe3c4eab186aaa33f04d7826c2239bfe99b3b392b026d18b8f31584652303R939]
> has changes StaticSegmentIterator to just SegmentIterator, which largely
> aligns with this patch, but might cause a rebase.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
