After couple of days it's still behaving fine. Case closed.
On Thu, Feb 19, 2015 at 11:15 PM, Michał Łowicki mlowi...@gmail.com wrote:
Upgrade to 2.1.3 seems to help so far. After ~12 hours total memory
consumption grew from 10GB to 10.5GB.
On Thu, Feb 19, 2015 at 2:02 PM, Carlos Rolo
In all tables SSTable counts is below 30.
On Thu, Feb 19, 2015 at 9:43 AM, Carlos Rolo r...@pythian.com wrote:
Can you check how many SSTables you have? It is more or less a know fact
that 2.1.2 has lots of problems with compaction so a upgrade can solve it.
But a high number of SSTables can
On Thu, Feb 19, 2015 at 10:41 AM, Carlos Rolo r...@pythian.com wrote:
So compaction doesn't seem to be your problem (You can check with nodetool
compactionstats just to be sure).
pending tasks: 0
How much is your write latency on your column families? I had OOM related
to this before,
So compaction doesn't seem to be your problem (You can check with nodetool
compactionstats just to be sure).
How much is your write latency on your column families? I had OOM related
to this before, and there was a tipping point around 70ms.
--
--
Do you have trickle_fsync enabled? Try to enable that and see if it solves
your problem, since you are getting out of non-heap memory.
Another question, is always the same nodes that die? Or is 2 out of 4 that
die?
Regards,
Carlos Juzarte Rolo
Cassandra Consultant
Pythian - Love your data
Then you are probably hitting a bug... Trying to find out in Jira. The bad
news is the fix is only to be released on 2.1.4. Once I find it out I will
post it here.
Regards,
Carlos Juzarte Rolo
Cassandra Consultant
Pythian - Love your data
rolo@pythian | Twitter: cjrolo | Linkedin:
Hi,
Couple of times a day 2 out of 4 members cluster nodes are killed
root@db4:~# dmesg | grep -i oom
[4811135.792657] [ pid ] uid tgid total_vm rss cpu oom_adj
oom_score_adj name
[6559049.307293] java invoked oom-killer: gfp_mask=0x201da, order=0,
oom_adj=0, oom_score_adj=0
Nodes are
On Wed, Feb 18, 2015 at 10:28 AM, Michał Łowicki mlowi...@gmail.com wrote:
Couple of times a day 2 out of 4 members cluster nodes are killed
This sort of issue is usually best handled/debugged interactively on IRC.
But briefly :
- 2.1.2 is IMO broken for production. Downgrade (officially
