Benedict created CASSANDRA-6916:
-----------------------------------
Summary: Preemptive re-open of compaction result
Key: CASSANDRA-6916
URL: https://issues.apache.org/jira/browse/CASSANDRA-6916
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Minor
Fix For: 2.1
Related to CASSANDRA-6812, but a little simpler: when compacting, we mess quite
badly with the page cache. One thing we can do to mitigate this problem is to
use the sstable we're writing before we've finished writing it, and to drop the
regions from the old sstables from the page cache as soon as the new sstables
have them (even if they're only written to the page cache). This should
minimise any page cache churn, as the old sstables must be larger than the new
sstable, and since both will be in memory, dropping the old sstables is at
least as good as dropping the new.
The approach is quite straight-forward. Every X MB written:
# grab flushed length of index file;
# grab second to last index summary record, after excluding those that point to
positions after the flushed length;
# open index file, and check that our last record doesn't occur outside of the
flushed length of the data file (pretty unlikely)
# Open the sstable with the calculated upper bound
Some complications:
# must keep running copy of compression metadata for reopening with
# we need to be able to replace an sstable with itself but a different lower
bound
# we need to drop the old page cache only when readers have finished
--
This message was sent by Atlassian JIRA
(v6.2#6252)
