Hello,
In the process of upgrading our cluster from 2.1 to 2.2 we have triggered
the SSTable rewriting process like this:
$ nodetool upgradesstables -j 4 # concurrent_compactors=5
Then if we would immediately check the compactionstats, we see that 4
compactions of type 'Upgrade sstables' are running for the same
keyspace/table. Fine.
What is surprising, though, is that when any of the tasks which are smaller
in size complete, no new tasks are added and we are left waiting for a
smaller number of long running tasks to complete, e.g:
pending tasks: 4
id compaction type keyspace
table completed total unit progress
d535e962-00de-11e8-9c2c-ade7b0b922e3 Upgrade sstables cel_data
event_history_unbounded 32.87 GB 84.12 GB bytes 39.08%
d535e960-00de-11e8-9c2c-ade7b0b922e3 Upgrade sstables cel_data
event_history_unbounded 33.15 GB 84.79 GB bytes 39.09%
Periodically, an ordinary compaction would be listed, but no new Upgrade
tasks appear.
Only after all of the "current" tasks are finished, the new 4 tasks are
scheduled and the process repeats.
We would expect that new tasks would be scheduled as soon as a compaction
slot is freed up, but this doesn't seem to happen. Is this by design? I
do not see any open issue about this in Jira.
Cheers,
--
Oleksandr "Alex" Shulgin | Database Engineer | Zalando SE | Tel: +49 176
127-59-707
