RE: Re: Unable to gossip with peers when starting cluster

0.9 was never a seed before. Based on your comment, I also tried, from having all three nodes up (following the initial bootstrap), restarting 0.7. This failed with the same error. On 2022/11/09 15:37:24 Jeff Jirsa wrote: > When you say you configured them to talk to .0.31 as a seed, did

Re: Unable to gossip with peers when starting cluster

When you say you configured them to talk to .0.31 as a seed, did you do that by changing the yaml? Was 0.9 ever a seed before? I expect if you start 0.7 and 0.9 at the same time, it all works. This looks like a logic/state bug that needs to be fixed, though. (If you're going to upgrade, usually

Re: Unable to Gossip

Oh!  Excellent!  Doh!  That was it.  So when we add a new system, we use puppet to push things out...like NTP...well this is our first Rocky Linux install and guess what I didn't do? Thank you Song.  The new machine is now joining the cluster. nodetool status Datacenter: datacenter1

Re: Unable to Gossip

Hello Joe, These logs indicate the clocks are out of sync (by over 4.2 hours) between the new node and the seed nodes: INFO  [ScheduledTasks:1] 2021-09-10 11:14:26,567 MessagingMetrics.java:206 - GOSSIP_DIGEST_SYN messages were dropped in last 5000 ms: 0 internal and 1 cross node.

Re: Unable to Gossip

Hmm. are the ports open on the `new` server? Looks like it can connect to other nodes but other nodes can't connect to it. -Vy On Fri, Sep 10, 2021 at 10:20 AM Joe Obernberger < joseph.obernber...@gmail.com> wrote: > Good idea. > There are two seed nodes: > I see this on one (note

Re: Unable to Gossip

Good idea. There are two seed nodes: I see this on one (note 172.16.100.44 is the new node): DEBUG [CompactionExecutor:1345] 2021-09-10 11:13:49,569 TimeWindowCompactionStrategy.java:129 - TWCS skipping check for fully expired SSTables INFO  [Messaging-EventLoop-3-10] 2021-09-10 11:14:22,810

Re: Unable to Gossip

Can you drop as much info as possible into a JIRA? Include the output of `nodetool gossipinfo` if at all possible On Fri, Sep 10, 2021 at 7:58 AM Joe Obernberger < joseph.obernber...@gmail.com> wrote: > Thank you Jeff - yes, this is on the latest 4.0.1 > > nodetool version > ReleaseVersion:

Re: Unable to Gossip

Thank you Jeff - yes, this is on the latest 4.0.1 nodetool version ReleaseVersion: 4.0.1 nodetool status Datacenter: datacenter1 === Status=Up/Down |/ State=Normal/Leaving/Joining/Moving --  Address Load    Tokens  Owns (effective)  Host ID

Re: Unable to Gossip

Is this on 4.0.0 ? 4.0.1 fixes an issue where the gossip result is too large for the urgent message queue, causing this stack trace, and was released 3 days ago. I've never seen it on a 10 node cluster before, but I'd be trying that. On Fri, Sep 10, 2021 at 7:50 AM Joe Obernberger <

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

For clarity for anybody that comes to this chain in the archive. This might be an issue with Ec2MultiRegionSnitch all together; not sure. But if I create a local 3 node cluster using ccm (cassandra v 3.11.4). I can drop the keystore/truststore jks files in, and flip encryption and everything

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

telnet from node 1 -> node2 7001 (and 7000) works. However, I can't rule out a JKS keystore/truststore issue. I have tried a number of configurations and none of them have seemed to help (or emit any further error logging). We have a root and intermediate CA cert, and a private key + signed

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

could be issue with keystore/trustore --- you may want to do keytool -- list  -- validate the files/password; also do md5sum on files from 1 node in west and 1 node in east.check ssl port 7001 --- from 1 node in west --> telnet :7001 (or custom port if you are not using default port) On

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

Subroto - both tools error; openssl errno 111 - which made me check bound ports on the c* node with encryption flipped. Port 9042 is not open (determined by netstat -ant). Looking at the log differences for when a node is started with/without encryption. Without encryption, I get a bunch of

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

The version given by apt is 8u162-b12-1. Which I think corresponds to openJDK-8-162. When I run jrunscript -e 'print (javax.crypto.Cipher.getMaxAllowedKeyLength("RC5") >= 256);' the command returns true. Not sure if that is the best way to verify JCE installed. Michael Carlise On Mon, Aug

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

Michael, Are you able to connect to any c* node via OpenSSL? Openssl s_client -connect :9042 Cqlsh —ssl Subroto > On Aug 26, 2019, at 2:47 PM, Marc Selwan wrote: > > which exact version of OpenJDK are you using? Is it possible you don't have > JCE on those nodes? (I believe more recent

Re: unable to gossip with peers exception when internode encryption is set to any setting other than 'none'

which exact version of OpenJDK are you using? Is it possible you don't have JCE on those nodes? (I believe more recent versions of Java 8 has this baked in so that might not be it) *Marc Selwan | *DataStax *| *PM, Server Team *|* *(925) 413-7079* *|* Twitter *

Re: Unable to gossip with any seeds

Hi Sundeep, Can you please confirm, are you configuring two nodes in different Datacenters? If you are configuring on single datacenter with two nodes,then please change the endpoint_snitch from RackInferringSnitch to SimpleSnitch and restart the clusters. Regards, Chiru On 03-Feb-2014, at

Re: Unable to gossip with any seeds

Thank you Chiru for the reply. I am configuring single datacenter. I changed it to SimpleSnitch. However, I am getting the same error. -Sundeep On Mon, Feb 3, 2014 at 3:58 AM, Chiranjeevi Ravilla rccassandr...@gmail.com wrote: Hi Sundeep, Can you please confirm, are you configuring two