Re: Speed up compaction

2019-06-13 Thread Alexander Dejanovski
Hi Léo, Major compactions in LCS (and minor as well) are very slow indeed and I'm afraid there's not much you can do to speed things up. There are lots of synchronized sections in the LCS code and it has to do a lot of comparisons between sstables to make sure a partition won't end up in two

Re: Speed up compaction

2019-06-13 Thread Léo FERLIN SUTTON
On Thu, Jun 13, 2019 at 2:44 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Thu, Jun 13, 2019 at 2:07 PM Léo FERLIN SUTTON > wrote: > >> >> Overall we are talking about a 1.08TB table, using LCS. >> >> SSTable count: 1047 >>> SSTables in each level: [15/4, 10, 103/100, 918, 0,

Re: Speed up compaction

2019-06-13 Thread Oleksandr Shulgin
On Thu, Jun 13, 2019 at 2:07 PM Léo FERLIN SUTTON wrote: > > Overall we are talking about a 1.08TB table, using LCS. > > SSTable count: 1047 >> SSTables in each level: [15/4, 10, 103/100, 918, 0, 0, 0, 0, 0] > > SSTable Compression Ratio: 0.5192269874287099 > > Number of partitions (estimate):

Re: Speed up compaction

2019-06-13 Thread Léo FERLIN SUTTON
On Thu, Jun 13, 2019 at 12:09 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Thu, Jun 13, 2019 at 11:28 AM Léo FERLIN SUTTON > wrote: > >> >> ## Cassandra configuration : >> 4 concurrent_compactors >> Current compaction throughput: 150 MB/s >> Concurrent reads/write are both

Re: Speed up compaction

2019-06-13 Thread Oleksandr Shulgin
On Thu, Jun 13, 2019 at 11:28 AM Léo FERLIN SUTTON wrote: > > ## Cassandra configuration : > 4 concurrent_compactors > Current compaction throughput: 150 MB/s > Concurrent reads/write are both set to 128. > > I have also temporarily stopped every repair operations. > > Any ideas about how I can

Speed up compaction

2019-06-13 Thread Léo FERLIN SUTTON
I am currently noticing very very slow compactions on my cluster and wondering if there is any way to speed things up. Right now I have this compaction currently running : 60c1cfc0-8da7-11e9-bc08-3546c703a280Compaction keyspace1 > table1 8.77 GB 1.71 TB bytes 0.50%