[
https://issues.apache.org/jira/browse/CASSANDRA-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347879#comment-14347879
]
Takenori Sato commented on CASSANDRA-8866:
------------------------------------------
Design notes from the first patch(cassandra-2.0-8866.txt)
The patch obviously lacks enough unit tests, but enough for design review. So
let me share. More unit tests will come soon.
h3.1. how to partition?
A partition is an equally divided range of a ring by a given IPartitioner. The
whole range(_new Range<Token>(min, min)_) is divided recursively to a given
partitions(16 by defaults). This is done by calling _IPartitioner#midpoint_
recursively, so the number of partitions has to be power of two.
h3.2 partitioned or unpartitioned?
A partitioned or an unpartitioned sstable is checked if a range(_new
Range<Token>(reader.first.getToken(), reader.last.getToken())_) of a sstable is
entirely contained in one of partition ranges.
If not, a PartitioningCompactionTask is generated to make partitions.
Otherwise, a CompactionTask is generated for the most interesting partition.
h3.3. how to put a row of unpartitioned sstables in partitioned sstables?
A PartitioningCompactionTask tracks in which range a compaction iterator walks.
And it tells a new segment is reached if it moves from one range to another.
For this purpose, I added one method to AbstractCompctionTask, which is
_newSSTableSegmentThresholdWillReach(AbstractCompactedRow nextRow)_. It is
called by a CompactionTask as _newSSTableSegmentThresholdReached_ is done, but
before writing a row.
h3.4. how to find an interesting partition?
This may require more considerations through real world tests. For now, an
interesting score is calculated as follows:
{code:borderStyle=solid}
int score = 0;
score += (numberOfSSTables >= minThreshold) ? ((numberOfSSTables >=
maxThreshold) ? 2 : 1) : 0;
score += (tombstones >
DatabaseDescriptor.getTombstoneWarnThreshold())
? ((tombstones >
DatabaseDescriptor.getTombstoneFailureThreshold()) ? 3: 1)
: 0;
{code}
> PartitionedCompactionStrategy
> -----------------------------
>
> Key: CASSANDRA-8866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8866
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Takenori Sato
> Attachments: cassandra-2.0-8866.txt
>
>
> PartitionedCompactionStrategy is a new compaction strategy with the following
> goals in mind:
> * Column tombstone removal effectiveness
> * Read performance
> As the name suggests, PartitionedCompactionStrategy actively splits
> un-partitioned sstables(newly flushed, imported, compaction strategy switch)
> into partitions by IPartitioner. The number of nodes will be configurable.
> Then, PartitionedCompactionStrategy finds an interesting partition at
> compaction based on the followings:
> - the number of sstables
> - the ratio of droppable tombstones
> - read hotness
> You may think this design looks similar to SizeTieredCompactionStrategy and
> LeveledCompactionStrategy, but the big difference is that a compaction by
> PartitionedCompactionStrategy is based on rows(a partitions). And this allows
> more effective column tombstone removal, and better read performance.
> Also note that this will not require any changes to the other components. So
> this is expected to be a purely pluggable compaction strategy.
> A possible implementation of
> _PertitionedCompactionStrategy#getNextBackgroundTask()_ is as follows:
> # find un-partitioned sstables
> # split un-partitioned sstables into partitiones
> # group all the sstables into partitions
> # find an interesting partition
> #* the number of sstables
> #* the number of droppable tombstones
> #* hotness
> # create a compaction task for the interesting bucket if found
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
