Re: Riak nodes constantly crashing

2016-10-27 Thread Alexander Sicular
Take a look at the AAE settings here: http://docs.basho.com/riak/kv/latest/using/cluster-operations/active-anti-entropy/ -Alexander @siculars http://siculars.posthaven.com Sent from my iRotaryPhone > On Oct 26, 2016, at 16:17, Steven Joseph wrote: > > I don't think

Re: Riak nodes constantly crashing

2016-10-27 Thread Steven Joseph
I don't think you should disable AAE, you can tune its frequency. Steven On Thu, 27 Oct 2016 03:50 Ricardo Mayerhofer wrote: > Yes, I'll check if the problem is the AAE! I will disable it and see the > results. > > Thanks Steven! > > On Tue, Oct 25, 2016 at 6:54 PM,

Re: Riak nodes constantly crashing

2016-10-26 Thread Ricardo Mayerhofer
Yes, I'll check if the problem is the AAE! I will disable it and see the results. Thanks Steven! On Tue, Oct 25, 2016 at 6:54 PM, Steven Joseph wrote: > Hi Ricardo, > > If you are using systemd might have to check LimitNOFILE for your units. > Active anti entropy runs

Re: Riak nodes constantly crashing

2016-10-26 Thread Steven Joseph
Hi Ricardo, If you are using systemd might have to check LimitNOFILE for your units. Active anti entropy runs periodically. Steven On Wed, 26 Oct 2016 04:36 Ricardo Mayerhofer wrote: > What's weird is that the node crashes every minute at the same second. Is > there

Re: Riak nodes constantly crashing

2016-10-25 Thread Ricardo Mayerhofer
What's weird is that the node crashes every minute at the same second. Is there anything Riak may be running every minute? On Mon, Oct 24, 2016 at 8:28 PM, Ricardo Mayerhofer wrote: > I'm also pasting the free -m: > > total used free shared

Re: Riak nodes constantly crashing

2016-10-24 Thread Ricardo Mayerhofer
I'm also pasting the free -m: total used free sharedbuffers cached Mem: 15039 14557482 0 37 4594 -/+ buffers/cache: 9925 5114 Swap:0 0 0 On Mon, Oct 24, 2016 at 8:24 PM,

Re: Riak nodes constantly crashing

2016-10-24 Thread Ricardo Mayerhofer
Hi Alexander, Thanks for your response. We use multi-backend with bitcask and leveldb. - File descriptors seems to be ok, at least the config. ubuntu@ip-10-2-58-5:/var/log/riak$ sudo su riak sudo: unable to resolve host ip-10-2-58-5 riak@ip-10-2-58-5:/var/log/riak$ ulimit -n 65535 - Memory

Re: Riak nodes constantly crashing

2016-10-24 Thread Alexander Sicular
Disk, memory or file descriptors would be my guess. Bitcask? On Monday, October 24, 2016, Ricardo Mayerhofer wrote: > Hi all, > I have a Riak 1.4 where the nodes seems to be constantly crashing. All 5 > nodes are affected. > > However it seems Riak manage to get them up

Riak nodes constantly crashing

2016-10-24 Thread Ricardo Mayerhofer
Hi all, I have a Riak 1.4 where the nodes seems to be constantly crashing. All 5 nodes are affected. However it seems Riak manage to get them up again. Any idea on whats going on? Erros logs below. Thanks. error.log ... 2016-10-24 21:57:29.185 [error] <0.24570.1174> CRASH REPORT Process