Re: going down from RF=3 to RF=2, repair constantly falls over with JVM OOM

2013-07-05 Thread Eric Stevens
The following setting is probably not a good idea: bloom_filter_fp_chance = 1.0 It would disable the bloom filters all together, and this setting doesn't have appreciably greater benefits over a setting of 0.1 (which has the advantage of saving you from disk I/O 90% of the time for keys which don'

Re: going down from RF=3 to RF=2, repair constantly falls over with JVM OOM

2013-07-04 Thread Alain RODRIGUEZ
@Michal: all true, a clean up would certainly remove a lot of useless data there, and I also advice Evan to do it. However, Evan may want to continue repairing his cluster as a routine operation an there is no reason a RF change shouldn't lead to this kind of issues. @Evan : With this amount of da

Re: going down from RF=3 to RF=2, repair constantly falls over with JVM OOM

2013-07-04 Thread MichaƂ Michalski
I don't think you need to run repair if you decrease RF. At least I wouldn't do it. In case of *decreasing* RF have 3 nodes containing some data, but only 2 of them should store them from now on, so you should rather run cleanup, instead of repair, toget rid of the data on 3rd replica. And I g

going down from RF=3 to RF=2, repair constantly falls over with JVM OOM

2013-07-04 Thread Evan Dandrea
Hi, We've made the mistake of letting our nodes get too large, now holding about 3TB each. We ran out of enough free space to have a successful compaction, and because we're on 1.0.7, enabling compression to get out of the mess wasn't feasible. We tried adding another node, but we think this may h