Ghost compaction process

[Arturas Raizys]

Hello,

I'm having problem there in 1 node I have continues compaction process
running and consuming CPU. nodetool tpstats show 1 compaction in
progress, but if I try to query system.compactions_in_progress table, I
see 0 records. This never ending compaction does slow down node and it
becomes laggy.
I'm willing to hire a contractor to solve this problem if anyone is
interested.


Cheers,
Arturas

Re: Ghost compaction process

[Arturas Raizys]

Hi,

 Does `nodetool comactionstats` show nothing running as well? Also, for
 posterity what are some details of the setup (C* version, etc.)?

`nodetool comactionstats` does not return anything, it just waits.
If I do enable DEBUG logging, I see this line poping up while executing
`nodetool compactionstats` :
DEBUG [RMI TCP Connection(1856)-127.0.0.1] 2015-06-08 09:29:46,043
LeveledManifest.java:680 - Estimating [0, 0, 0, 0, 0, 0, 0, 0, 0]
compactions to do for system.paxos

I'm running Cassandra 2.1.14 with 7 node cluster. We're using small VM
with 8GB of ram and SSD. Our data size per node with RF=2 is ~40GB. Load
is ~ 1000 writes/second. Most of the data TTL is 2weeks.


Cheers,
Arturas