Hi All, We have a new observation.
Earlier for implementing multiple network interfaces, we were deleting cassandra-topologies.properties in the last step (Steps are mentioned in mail trail). The rationale was that because we are using altogether a new endpoint_snitch , we don't require cassandra-topologies.properties file anymore. Now we have observed that if we don't delete cassandra-topologies.properties, the slowness is not there in the cluster (Even with multiple restarts) Is there some relationship between GossipingPropertyFileSnitch and cassandra-topologies.properties ? As per my knowledge, cassandra-topologies.properties file is only used as a fallback while doing snitch migration. If that's the case, why does Cassandra becomes slow with time ( and after doing multiple restarts ) after deleting cassandra-topologies.properties ? Regards, Prakash Chauhan. From: Cogumelos Maravilha [mailto:cogumelosmaravi...@sapo.pt] Sent: Wednesday, May 24, 2017 12:15 AM To: u...@cassandra.apache.org Subject: Re: Slowness in C* cluster after implementing multiple network interface configuration. Hi, I never used version 2.0.x but I think port 7000 isn't enough. Try enable: 7000 inter-node 7001 SSL inter-node 9042 CQL 9160 Thrift is enable in that version And In Cassandra.yaml, add property "broadcast_address". = local ipv4 In Cassandra.yaml, change "listen_address" to private IP. = local ipv4 As a starting point. Cheers. On 22-05-2017 12:36, Prakash Chauhan wrote: Hi All , Need Help !!! Setup Details: Cassandra 2.0.14 Geo Red setup * DC1 - 3 nodes * DC2 - 3 nodes We were trying to implement multiple network interfaces with Cassandra 2.0.14 After doing all the steps mentioned in DataStax doc http://docs.datastax.com/en/archived/cassandra/2.0/cassandra/configuration/configMultiNetworks.html, we observed that nodes were not able to see each other (checked using nodetool status). To resolve this issue, we followed the comment<https://issues.apache.org/jira/browse/CASSANDRA-9748?focusedCommentId=14903515&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14903515> mentioned in the JIRA : CASSANDRA-9748<https://issues.apache.org/jira/browse/CASSANDRA-9748> Exact steps that we followed are : -------------------------------- 1. Stop Cassandra 2. Add rule to "iptables" to forward all packets on the public interface to the private interface. COMMAND: # iptables -t nat -A PREROUTING -p tcp -m tcp -d <destination> --dport 7000 -j DNAT --to-destination <to-destination>:7000 3. In Cassandra.yaml, add property "broadcast_address". 4. In Cassandra.yaml, change "listen_address" to private IP. 5. Clear the data from directory "peers". 6. Change Snitch to GossipingPropertyFileSnitch. 7. Append following property to the file "/etc/cassandra/conf/cassandra-env.sh" to purge gossip state. JVM_OPTS="$JVM_OPTS -Dcassandra.load_ring_state=false" 8. Start Cassandra 9. After node has been started, remove following property from the file "/etc/cassandra/conf/cassandra-env.sh" (previously added in step 7) JVM_OPTS="$JVM_OPTS -Dcassandra.load_ring_state=false" 10. Delete file "/etc/cassandra/conf/cassandra-topology.properties" Now We have an observation that after multiple restarts of Cassandra on multiple nodes, slowness is observed in the cluster. The problem gets resolved when we revert the steps mentioned above. Do u think there is any step that can cause the problem ? We are suspecting Step 2(iptable rule) but not very sure about it. Regards, Prakash Chauhan.
