[
https://issues.apache.org/jira/browse/CASSANDRA-10270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735431#comment-14735431
]
Vidur Malik commented on CASSANDRA-10270:
-----------------------------------------
On a related note:
We're running a Cassandra 2.2.0 cluster with 8 nodes. We are doing frequent
updates to our data and we have very few reads, and we are using Leveled
Compaction with a sstable_size_in_mb of 160MB. We don't have that much data
currently since we're just testing the cluster.
We are seeing the SSTable count linearly increase even though `nodetool
compactionhistory` shows that compactions have definitely run. When I ran
nodetool cfstats, I get the following output:
Table: tender_summaries
SSTable count: 56
SSTables in each level: [1, 0, 0, 0, 0, 0, 0, 0, 0]
Does it make sense that there is such a huge difference between the number of
SStables in each level and the total count of SStables? It seems like old
SSTables are lying around and never cleaned-up/compacted.
Schema is relatively simple:
CREATE TABLE IF NOT EXISTS reporting.tender_summaries (
organization_id uuid,
date timestamp,
year int,
location_id varchar,
operation_type varchar,
reference_id varchar,
field1 int,
field2 int,
field3 int,
field4 int,
PRIMARY KEY((organization_id, year), location_id, date, operation_type,
reference_id)
) WITH CLUSTERING ORDER BY (location_id DESC, date DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'sstable_size_in_mb': '160', 'class':
'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
> Cassandra stops compacting
> --------------------------
>
> Key: CASSANDRA-10270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10270
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: linux (google cloud click-to-deploy, default settings, 3
> nodes)
> Reporter: Adam Bliss
> Assignee: Marcus Eriksson
> Fix For: 2.2.x
>
> Attachments: system.log.gz, system.txt.gz
>
>
> My cassandra cluster won't keep compacting. I notice that if I restart, it
> does compact for a while, but after a time it stops. As a result, after
> adding a bunch of rows, I ended up with about 1000 sstables per node.
> I'll attach more logs in a minute, but it seems like this might be the most
> relevant part:
> {noformat}
> INFO [CompactionExecutor:1] 2015-09-04 14:22:55,796
> CompactionManager.java:1433 - Compaction interrupted:
> Compaction@fff9bcd0-3b1f-11e5-8df6-33158d7bf3bf(megacrawl2, ranks_by_domain,
> 812501702/7543091905)bytes
> DEBUG [CompactionExecutor:1] 2015-09-04 14:22:55,797
> CompactionManager.java:1437 - Full interruption stack trace:
> org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction
> interrupted: Compaction@fff9bcd0-3b1f-11e5-8df6-33158d7bf3bf(megacrawl2,
> ranks_by_
> domain, 812501702/7543091905)bytes
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:180)
> ~[apache-cassandra-2.2.1.jar:2.2.1]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.2.1.jar:2.2.1]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
> ~[apache-cassandra-2.2.1.jar:2.2.1]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[apache-cassandra-2.2.1.jar:2.2.1] at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:236)
> ~[apache-cassandra-2.2.1.jar:2.2.1]
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_79]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> DEBUG [CompactionExecutor:1] 2015-09-04 14:22:55,797
> CompactionManager.java:222 - Checking system.local
> DEBUG [CompactionExecutor:1] 2015-09-04
> 14:22:55,797 SizeTieredCompactionStrategy.java:85 - Compaction buckets are
> [[BigTableReader(path='/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/la-83-big-Data.db'),
>
> BigTableReader(path='/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/la-81-big-Data.db'),
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
