Re: how to increase compaction rate?

2012-03-13 Thread Thorsten von Eicken
On 3/12/2012 6:52 AM, Brandon Williams wrote: On Mon, Mar 12, 2012 at 4:44 AM, aaron morton aa...@thelastpickle.com wrote: I don't understand why I don't get multiple concurrent compactions running, that's what would make the biggest performance difference. concurrent_compactors Controls

Re: how to increase compaction rate?

2012-03-13 Thread Viktor Jevdokimov
After loosing one node we had to repair, CFs was on leveled compaction. For one CF each node had about 7GB of data. Running a repair without primary range switch ended up some nodes exhausted to about 60-100GB of 5MB sstables for that CF (a lot of files). After switching back from leveled to

Re: how to increase compaction rate?

2012-03-13 Thread Thorsten von Eicken
On 3/13/2012 4:13 PM, Viktor Jevdokimov wrote: What we did to speedup this process to return all exhausted nodes into normal state faster: We have created a 6 temporary virtual single Cassandra nodes with 2 CPU cores and 8GB RAM. Stopped completely a compaction for CF on a production node.

Re: how to increase compaction rate?

2012-03-13 Thread Edward Capriolo
On Tue, Mar 13, 2012 at 11:32 PM, Thorsten von Eicken t...@rightscale.com wrote: On 3/13/2012 4:13 PM, Viktor Jevdokimov wrote: What we did to speedup this process to return all exhausted nodes into normal state faster: We have created a 6 temporary virtual single Cassandra nodes with 2 CPU

Re: how to increase compaction rate?

2012-03-12 Thread Thorsten von Eicken
On 3/11/2012 9:17 PM, Peter Schuller wrote: multithreaded_compaction: false Set to true. I did try that. I didn't see it go any faster. The cpu load was lower, which I assumed meant fewer bytes/sec being compressed (SnappyCompressor). I didn't see multiple compactions in parallel. Nodetool

Re: how to increase compaction rate?

2012-03-12 Thread aaron morton
I don't understand why I don't get multiple concurrent compactions running, that's what would make the biggest performance difference. concurrent_compactors Controls how many concurrent compactions to run, by default it's the number of cores on the machine. If you are not CPU bound check

Re: how to increase compaction rate?

2012-03-12 Thread Brandon Williams
On Mon, Mar 12, 2012 at 4:44 AM, aaron morton aa...@thelastpickle.com wrote: I don't understand why I don't get multiple concurrent compactions running, that's what would make the biggest performance difference. concurrent_compactors Controls how many concurrent compactions to run, by

Re: how to increase compaction rate?

2012-03-11 Thread Peter Schuller
multithreaded_compaction: false Set to true. -- / Peter Schuller (@scode, http://worldmodscode.wordpress.com)