Milan Majercik created CASSANDRA-12673:

             Summary: Nodes cannot see each other in multi-DC, non-EC2 
environment with two-interface nodes due to outbound node-to-node connection 
binding to private interface
                 Key: CASSANDRA-12673
             Project: Cassandra
          Issue Type: Improvement
          Components: Core
         Environment: Multi-DC, non-EC2 environment with two-interface nodes
            Reporter: Milan Majercik
            Priority: Minor

We have a two-DC cluster in non-EC2 environment with each node containing two 
interfaces, one using private addresses for intra-DC communication and the 
other using public addresses for inter-DC communication. After proper 
configuration setup needed for this kind of environment we observed nodes 
cannot see each other.

The configuration changes made for this purpose are as follows:

listen_address: bound to private interface
broadcast_address: bound to public address
listen_on_broadcast_address: true
endpoint_snitch: GossipingPropertyFileSnitch
prefer_local=true (in

Upon restart, cassandra node contacts other nodes with their public addresses 
which is essential for making contacts to foreign data centers. After 
exhaustive investigation we found cassandra binds outbound node-to-node 
connections to private interface (the one specified in listen_address) that 
poses a problem for our environment as these data centers do not allow 
connections from private interface to public network.

A portion of cassandra code responsible for local binding of outbound 
connections can be found in method

            if (!Config.getOutboundBindAny())
InetSocketAddress(FBUtilities.getLocalAddress(), 0));

After we commented out these two lines and deployed cassandra.jar across the 
cluster, the nodes were able to see each other and everything appears to be 
working fine, including two-DC setup.

Do you think it's possible to remove these two lines without negative 
consequences? Alternatively, if the local binding serves some specific purpose 
of which I'm ignorant would it be possible to make it configurable?

This message was sent by Atlassian JIRA

Reply via email to