The Datastax doc says to run cleanup one node at a time after bootstrapping
has completed. The myadventuresincoding post says to run a repair on each
node first. Is it necessary to run the repairs first? Thanks.
On Tue, Apr 4, 2023 at 1:11 PM Bowen Song via user <
user@cassandra.apache.org>
In this case the removal process has already finished hours before, so
nothing is streaming anymore (but looking at the list of the transaction
logs left behind, could be some of the transaction finished while
decommission was still running).
We don't have secondary indexes,and no repairs were
You will DEFINITELY not remove sstables obsoleted by compaction if they are
being streamed out to neighbors. It would also not surprise me that if you
have something holding a background reference to one of the sstables in the
oldest/older compaction transaction logs, that the whole process may
If it was just one instance I would just bounce it but the thing is this
happens when we join/remove nodes, and we have a lot of servers with this
issue (while before the join/remove we are on ~50% disk usage).
We found ourselves fighting with compaction to make sure we don't run out
of space.
If you restart the node, it'll process/purge those compaction logs on
startup, but you want them to purge/process now.
I genuinely dont know when the tidier runs, but it's likely the case that
you're too busy compaction to purge (though I don't know what exactly "too
busy" means).
Since you're
More information - from another node in the cluster
I can see many txn files although I only have two compactions running.
[user@server808 new_table-44263b406bf111ed8bd9df80ace3677a]# ls -l *txn*
-rw-r--r-- 1 cassandra cassandra 613 Apr 4 05:26
Thanks. I also found this useful:
https://myadventuresincoding.wordpress.com/2020/08/03/cassandra-how-to-add-a-new-node-to-an-existing-cluster/
The node seems to be joining fine and is streaming in lots of data. Cluster
is still operating normally.
On Tue, Apr 4, 2023 at 1:11 PM Bowen Song via
Perhaps have a read here?
https://docs.datastax.com/en/cassandra-oss/3.x/cassandra/operations/opsAddNodeToCluster.html
On 04/04/2023 06:41, David Tinker wrote:
Ok. Have to psych myself up to the add node task a bit. Didn't go well
the first time round!
Tasks
- Make sure the new node is not
Hey
I have a 4.0.7 cluster in which I see weird behavior.
I expect that once compaction finishes, the old sstables that were part of
the compaction set will be deleted, but it appears they are deleted much
later, thus causing space issues.
For example this is what I have in the log, only one
