Re: [akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

On Friday, March 2, 2018 at 11:01:11 PM UTC+2, Manuel Bernhardt wrote: > > Hi, > > 6 seconds is quite a bit, but then again it sounds like you may have long > major GCs? > > If you have a network that doesn’t play nice, I’d suggest setting the > suspicion threshold at 12. You’ll get a slower de

Re: [akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

Hi, 6 seconds is quite a bit, but then again it sounds like you may have long major GCs? If you have a network that doesn’t play nice, I’d suggest setting the suspicion threshold at 12. You’ll get a slower detection time but with 6 seconds of shift you don’t get that anyway. And you’ll have fever

[akka-user] Re: akka http 10.0.11 under OSGi container

I brought this up before for 10.0.11, but forgot to open the PR like was suggested! It appears OSGi support was removed in 10.1.0-RC1, though they are actively seeking help. On

Re: [akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

On Friday, March 2, 2018 at 6:05:11 PM UTC+2, Patrik Nordwall wrote: > > Sounds interesting/strange. For investigation it would be good with debug > logging and verbose-heartbeat-logging. > > I think it is config akka.cluster.debug.verbose-heartbeat-logging=on, but > check reference.conf > I

Re: [akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

On Friday, March 2, 2018 at 4:47:13 PM UTC+2, Manuel Bernhardt wrote: > > How big is your cluster? > We have 100s of clusters on all environments, issue manifests on the large ones which top ~ 20 machines or so (not that huge). > > It looks like the failure detector takes much longer than yo

Re: [akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

Sounds interesting/strange. For investigation it would be good with debug logging and verbose-heartbeat-logging. I think it is config akka.cluster.debug.verbose-heartbeat-logging=on, but check reference.conf /Patrik fre 2 mars 2018 kl. 15:47 skrev Manuel Bernhardt : > How big is your cluster? >

Re: [akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

How big is your cluster? It looks like the failure detector takes much longer than you'd want it to (or simply does not) to no longer suspect other nodes. This could happen with an accrual FD that gets a lot of slow heartbeats. What's your value of the suspicion threshold? (akka.cluster.failure-de

[akka-user] Akka cluster - issues with nodes getting stuck to Unreachable permanently

Hi all, First off all, i'd like to state that we have a "noisy" operational environment where network partitions occur more often than we'd like and certain components (cluster nodes) experience high GC pause times. That being said, we are facing the following issue with a higher frequency tha