[
https://issues.apache.org/jira/browse/CASSANDRA-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863553#action_12863553
]
Jonathan Ellis commented on CASSANDRA-1041:
-------------------------------------------
You don't want to skip large files in major compactions, since the definition
of major is "compact everything so it is safe to remove tombstones." (See
http://wiki.apache.org/cassandra/DistributedDeletes and
http://wiki.apache.org/cassandra/MemtableSSTable).
Major compactions are not run automatically so if you don't want that going on
just don't invoke nodeprobe compact.
Minor compaction are batched up according to sstable size and only similar
sizes are merged. Here again ignoring larger sstables indefinitely is a bad
idea, because the number of sstables you have to merge rows from for reads is
unbounded.
Better solution: expose CompactionManager.minimumCompactionThreshold and
CM.maximumCompactionThreshold in the configuration file, so you can tune minor
compactions to require more or less sstables in the same "bucket" before
compacting.
> Skip large size (Configurable) SSTable in minor or/and major compaction
> -----------------------------------------------------------------------
>
> Key: CASSANDRA-1041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1041
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Schubert Zhang
> Priority: Minor
> Attachments: CASSANDRA-1041-0.6.1.patch, CASSANDRA-1041-0.6.patch
>
>
> When the SSTable files are large enough, such as 100GB, the compaction
> (include minor and major) cost is big (disk IO, CPU, memory), etc.
> In some applications, we accept not compcating all SSTables to the final very
> large ones.
> This feature provide two optional configurable attributes
> MinorCompactSkipInGB and MajorCompactSkipInGB for each ColumnFamily.
> The optional MinorCompactSkipInGB attribute specifies the maximum size of
> SSTables which will be compcated in minor-compaction. The SSTables larger
> than MinorCompactSkipInGB will be skipped. The optional MajorCompactSkipInGB
> attribute is same for major-compaction.
> The default of these attributes are 0, means do not skip, just as current
> 0.6.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
