[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-12673:
Resolution: Fixed
Fix Version/s: (was: 3.0.x)
(was: 2.2.x)
(was: 3.x)
3.10
3.0.11
2.2.9
Status: Resolved (was: Ready to Commit)
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Assignee: Milan Majercik
>Priority: Minor
> Fix For: 2.2.9, 3.0.11, 3.10
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-12673:
Status: Ready to Commit (was: Patch Available)
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Assignee: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-12673:
Assignee: Milan Majercik
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Assignee: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12673:
Issue Type: Bug (was: Improvement)
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Milan Majercik updated CASSANDRA-12673:
---
Fix Version/s: 3.x
3.0.x
Status: Patch Available (was: Open)
Issued patches according to suggestion of [~pauloricardomg].
* [2.2|https://github.com/mmajercik/cassandra/tree/12673-2.2]
* [3.0|https://github.com/mmajercik/cassandra/tree/12673-3.0]
* [3.X|https://github.com/mmajercik/cassandra/tree/12673-3.X]
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-12673:
Status: Open (was: Patch Available)
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-12673:
Reviewer: Paulo Motta
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Milan Majercik updated CASSANDRA-12673:
---
Fix Version/s: 2.2.x
Status: Patch Available (was: Open)
Eliminated local bind for outbound connections
[https://github.com/mmajercik/cassandra/tree/12673-2.2]
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
> Fix For: 2.2.x
>
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
> *endpoint_snitch*: GossipingPropertyFileSnitch
> *prefer_local*=true (in cassandra-rackdc.properties)
> Upon restart, cassandra node contacts other nodes with their public addresses
> which is essential for making contacts to foreign data centers. After
> exhaustive investigation we found cassandra binds outbound node-to-node
> connections to private interface (the one specified in listen_address) that
> poses a problem for our environment as these data centers _do not allow
> connections from private interface to public network_.
> A portion of cassandra code responsible for local binding of outbound
> connections can be found in method
> {{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
> {code}
> if (!Config.getOutboundBindAny())
> channel.bind(new
> InetSocketAddress(FBUtilities.getLocalAddress(), 0));
> {code}
> After we commented out these two lines and deployed cassandra.jar across the
> cluster, the nodes were able to see each other and everything appears to be
> working fine, including two-DC setup.
> Do you think it's possible to remove these two lines without negative
> consequences? Alternatively, if the local binding serves some specific
> purpose of which I'm ignorant would it be possible to make it configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Milan Majercik updated CASSANDRA-12673:
---
Description:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers _do not allow
connections from private interface to public network_.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
{{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}:
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code}
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
was:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers _do not allow
connections from private interface to public network_.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
{{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code}
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
>
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Milan Majercik updated CASSANDRA-12673:
---
Description:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers _do not allow
connections from private interface to public network_.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
{{org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket}}
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code}
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
was:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers do not allow
connections from private interface to public network.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code}
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
>
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Milan Majercik updated CASSANDRA-12673:
---
Description:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers do not allow
connections from private interface to public network.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code}
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
was:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers do not allow
connections from private interface to public network.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code|
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
>
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[
https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Milan Majercik updated CASSANDRA-12673:
---
Description:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers do not allow
connections from private interface to public network.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket
{code}
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
{code|
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
was:
We have a two-DC cluster in non-EC2 environment with each node containing two
interfaces, one using private addresses for intra-DC communication and the
other using public addresses for inter-DC communication. After proper
configuration setup needed for this kind of environment we observed nodes
cannot see each other.
The configuration changes made for this purpose are as follows:
*listen_address*: bound to private interface
*broadcast_address*: bound to public address
*listen_on_broadcast_address*: true
*endpoint_snitch*: GossipingPropertyFileSnitch
*prefer_local*=true (in cassandra-rackdc.properties)
Upon restart, cassandra node contacts other nodes with their public addresses
which is essential for making contacts to foreign data centers. After
exhaustive investigation we found cassandra binds outbound node-to-node
connections to private interface (the one specified in listen_address) that
poses a problem for our environment as these data centers do not allow
connections from private interface to public network.
A portion of cassandra code responsible for local binding of outbound
connections can be found in method
org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket
if (!Config.getOutboundBindAny())
channel.bind(new
InetSocketAddress(FBUtilities.getLocalAddress(), 0));
After we commented out these two lines and deployed cassandra.jar across the
cluster, the nodes were able to see each other and everything appears to be
working fine, including two-DC setup.
Do you think it's possible to remove these two lines without negative
consequences? Alternatively, if the local binding serves some specific purpose
of which I'm ignorant would it be possible to make it configurable?
> Nodes cannot see each other in multi-DC, non-EC2 environment with
> two-interface nodes due to outbound node-to-node connection binding to
> private interface
> --
>
> Key: CASSANDRA-12673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12673
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Multi-DC, non-EC2 environment with two-interface nodes
>Reporter: Milan Majercik
>Priority: Minor
>
> We have a two-DC cluster in non-EC2 environment with each node containing two
> interfaces, one using private addresses for intra-DC communication and the
> other using public addresses for inter-DC communication. After proper
> configuration setup needed for this kind of environment we observed nodes
> cannot see each other.
> The configuration changes made for this purpose are as follows:
> *listen_address*: bound to private interface
> *broadcast_address*: bound to public address
> *listen_on_broadcast_address*: true
>
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[ https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Majercik updated CASSANDRA-12673: --- Description: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: *listen_address*: bound to private interface *broadcast_address*: bound to public address *listen_on_broadcast_address*: true *endpoint_snitch*: GossipingPropertyFileSnitch *prefer_local*=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? was: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: *listen_address*: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? > Nodes cannot see each other in multi-DC, non-EC2 environment with > two-interface nodes due to outbound node-to-node connection binding to > private interface > -- > > Key: CASSANDRA-12673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12673 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Multi-DC, non-EC2 environment with two-interface nodes >Reporter: Milan Majercik >Priority: Minor > > We have a two-DC cluster in non-EC2 environment with each node containing two > interfaces, one using private addresses for intra-DC communication and the > other using public addresses for inter-DC communication. After proper > configuration setup needed for this kind of environment we observed nodes > cannot see each other. > The configuration changes made for this purpose are as follows: > *listen_address*: bound to private interface > *broadcast_address*: bound to public address > *listen_on_broadcast_address*: true > *endpoint_snitch*:
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[ https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Majercik updated CASSANDRA-12673: --- Description: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: *listen_address*: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? was: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: listen_address: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? > Nodes cannot see each other in multi-DC, non-EC2 environment with > two-interface nodes due to outbound node-to-node connection binding to > private interface > -- > > Key: CASSANDRA-12673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12673 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Multi-DC, non-EC2 environment with two-interface nodes >Reporter: Milan Majercik >Priority: Minor > > We have a two-DC cluster in non-EC2 environment with each node containing two > interfaces, one using private addresses for intra-DC communication and the > other using public addresses for inter-DC communication. After proper > configuration setup needed for this kind of environment we observed nodes > cannot see each other. > The configuration changes made for this purpose are as follows: > *listen_address*: bound to private interface > broadcast_address: bound to public address > listen_on_broadcast_address: true > endpoint_snitch:
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[ https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Majercik updated CASSANDRA-12673: --- Description: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: listen_address: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? was: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: listen_address: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? > Nodes cannot see each other in multi-DC, non-EC2 environment with > two-interface nodes due to outbound node-to-node connection binding to > private interface > -- > > Key: CASSANDRA-12673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12673 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Multi-DC, non-EC2 environment with two-interface nodes >Reporter: Milan Majercik >Priority: Minor > > We have a two-DC cluster in non-EC2 environment with each node containing two > interfaces, one using private addresses for intra-DC communication and the > other using public addresses for inter-DC communication. After proper > configuration setup needed for this kind of environment we observed nodes > cannot see each other. > The configuration changes made for this purpose are as follows: > listen_address: bound to private interface > broadcast_address: bound to public address > listen_on_broadcast_address: true > endpoint_snitch:
[jira] [Updated] (CASSANDRA-12673) Nodes cannot see each other in multi-DC, non-EC2 environment with two-interface nodes due to outbound node-to-node connection binding to private interface
[ https://issues.apache.org/jira/browse/CASSANDRA-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Majercik updated CASSANDRA-12673: --- Description: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: listen_address: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? was: We have a two-DC cluster in non-EC2 environment with each node containing two interfaces, one using private addresses for intra-DC communication and the other using public addresses for inter-DC communication. After proper configuration setup needed for this kind of environment we observed nodes cannot see each other. The configuration changes made for this purpose are as follows: listen_address: bound to private interface broadcast_address: bound to public address listen_on_broadcast_address: true endpoint_snitch: GossipingPropertyFileSnitch prefer_local=true (in cassandra-rackdc.properties) Upon restart, cassandra node contacts other nodes with their public addresses which is essential for making contacts to foreign data centers. After exhaustive investigation we found cassandra binds outbound node-to-node connections to private interface (the one specified in listen_address) that poses a problem for our environment as these data centers do not allow connections from private interface to public network. A portion of cassandra code responsible for local binding of outbound connections can be found in method org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket if (!Config.getOutboundBindAny()) channel.bind(new InetSocketAddress(FBUtilities.getLocalAddress(), 0)); After we commented out these two lines and deployed cassandra.jar across the cluster, the nodes were able to see each other and everything appears to be working fine, including two-DC setup. Do you think it's possible to remove these two lines without negative consequences? Alternatively, if the local binding serves some specific purpose of which I'm ignorant would it be possible to make it configurable? > Nodes cannot see each other in multi-DC, non-EC2 environment with > two-interface nodes due to outbound node-to-node connection binding to > private interface > -- > > Key: CASSANDRA-12673 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12673 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Multi-DC, non-EC2 environment with two-interface nodes >Reporter: Milan Majercik >Priority: Minor > > We have a two-DC cluster in non-EC2 environment with each node containing two > interfaces, one using private addresses for intra-DC communication and the > other using public addresses for inter-DC communication. After proper > configuration setup needed for this kind of environment we observed nodes > cannot see each other. > The configuration changes made for this purpose are as follows: > listen_address: bound to private interface > broadcast_address: bound to public address > listen_on_broadcast_address: true > endpoint_snitch:
