Then you are probably hitting a bug... Trying to find out in Jira. The bad news is the fix is only to be released on 2.1.4. Once I find it out I will post it here.
Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo <http://linkedin.com/in/carlosjuzarterolo>* Tel: 1649 www.pythian.com On Thu, Feb 19, 2015 at 12:16 PM, Michał Łowicki <mlowi...@gmail.com> wrote: > |trickle_fsync| has been enabled for long time in our settings (just > noticed): > > trickle_fsync: true > > trickle_fsync_interval_in_kb: 10240 > > On Thu, Feb 19, 2015 at 12:12 PM, Michał Łowicki <mlowi...@gmail.com> > wrote: > >> >> >> On Thu, Feb 19, 2015 at 11:02 AM, Carlos Rolo <r...@pythian.com> wrote: >> >>> Do you have trickle_fsync enabled? Try to enable that and see if it >>> solves your problem, since you are getting out of non-heap memory. >>> >>> Another question, is always the same nodes that die? Or is 2 out of 4 >>> that die? >>> >> >> Always the same nodes. Upgraded to 2.1.3 two hours ago so we'll monitor >> if maybe issue has been fixed there. If not will try to enable >> |tricke_fsync| >> >> >>> >>> Regards, >>> >>> Carlos Juzarte Rolo >>> Cassandra Consultant >>> >>> Pythian - Love your data >>> >>> rolo@pythian | Twitter: cjrolo | Linkedin: >>> *linkedin.com/in/carlosjuzarterolo >>> <http://linkedin.com/in/carlosjuzarterolo>* >>> Tel: 1649 >>> www.pythian.com >>> >>> On Thu, Feb 19, 2015 at 10:49 AM, Michał Łowicki <mlowi...@gmail.com> >>> wrote: >>> >>>> >>>> >>>> On Thu, Feb 19, 2015 at 10:41 AM, Carlos Rolo <r...@pythian.com> wrote: >>>> >>>>> So compaction doesn't seem to be your problem (You can check with >>>>> nodetool compactionstats just to be sure). >>>>> >>>> >>>> pending tasks: 0 >>>> >>>> >>>>> >>>>> How much is your write latency on your column families? I had OOM >>>>> related to this before, and there was a tipping point around 70ms. >>>>> >>>> >>>> Write request latency is below 0.05 ms/op (avg). Checked with OpsCenter. >>>> >>>> >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> BR, >>>> Michał Łowicki >>>> >>> >>> >>> -- >>> >>> >>> >>> >> >> >> -- >> BR, >> Michał Łowicki >> > > > > -- > BR, > Michał Łowicki > -- --
