[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923317#comment-13923317
]
Jason Brown commented on CASSANDRA-5899:
Took a long time to think through this one, but I think on the whole it is fine
(even the EC2MRS change).
One (minor) thing to consider, however, is that with ec2 regions, typically the
client will only want to connect to server nodes within the same region and
then on the private IP addr. Your change sets the broadcastRpcAddr to the
public IP addr in EC2MRS. This has drawbacks as then clients will need to have
ingress privs set on the new server's security group (even if it is in the same
region!) to even connect to the node. The opposite of this is using the private
IP addr for the broadcast addr, which optimizes local region communication, but
is absolutely will not work for connecting to another region. So, I suppose
something you could try is:
- assume all clients connected to a given server are only in the local region
(as the server)
- if peer server (that you will send notifications about) is in the same
region, send the private IP addr (you can find it in
ApplicationState.INTERNAL_IP, more or less)
- else send the public IP addr (broadcastRpcAddr in your patch)
I'm not sure we can make the assumption all connected clients will be in the
same region as the server (which is sending it notifications), so then the only
logical choice is to send the public IP addr to the client. The client can then
do some machinations to find out about the server for which it got the IP addr,
and then potentially switch to the private IP addr if appropriate.
All this to say, +1 as is.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Priority: Minor
Fix For: 2.1 beta2
Attachments: 5899.txt
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13916128#comment-13916128
]
Tyler Hobbs commented on CASSANDRA-5899:
Sure, I'll take it.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Priority: Minor
Fix For: 2.1 beta2
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13913495#comment-13913495
]
Tyler Hobbs commented on CASSANDRA-5899:
bq. The fact that we don't know which address to send when rpc_address is
0.0.0.0 is a slightly different problem, but we could make it mandatory (or
strongly recommended, but I'd personally would vote for mandatory) to set a
broadcast_rpc_address if rpc_address=0.0.0.0 and kill two birds with one stone.
I think requiring broadcast_rpc_address to be set if rpc_address==0.0.0.0 is
reasonable, but we should get that done shortly if we want it in 2.1. Are you
going to have time to do this?
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.1 beta2
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914182#comment-13914182
]
Sylvain Lebresne commented on CASSANDRA-5899:
-
bq. Are you going to have time to do this?
Not sure, but I can definitively find some time to review if you want to tackle
it :)
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.1 beta2
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847316#comment-13847316
]
Sylvain Lebresne commented on CASSANDRA-5899:
-
bq. Why don't we add an broadcast_rpc_address config option?
Good idea. I mean, I think we need it anyway for the same reasons that we have
a broadcast_listen_address, that is for when none of the interfaces on the node
are actually public to the client because there is some router in between.
The fact that we don't know which address to send when rpc_address is 0.0.0.0
is a slightly different problem, but we could make it mandatory (or strongly
recommended, but I'd personally would vote for mandatory) to set a
broadcast_rpc_address if rpc_address=0.0.0.0 and kill two birds with one stone.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.1
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847477#comment-13847477
]
Chris Burroughs commented on CASSANDRA-5899:
So I don't think this is terribly uncommon, but as an example all of our nodes
have two interfaces. Depending on where a client is in the network there are 3
address (interfaces + one is NATed for inter-DC) that could potential work, but
only 1-2 that would work. Some might appear to work but would be very bad (go
out the local net and back in or something). I am unsure how any single value
for broadcast_rpc_address would be useful, nor how even a list could work since
which to choose is client location dependent.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.1
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847535#comment-13847535
]
Russell Bradberry commented on CASSANDRA-5899:
--
{quote}
I am unsure how any single value for broadcast_rpc_address would be useful
{quote}
A very common setup is to have a CNAME address that points to the internal IP
address when within the DC and an external IP address when outside the DC.
Setting the broadcast address to this common CNAME would allow clients both
internal and external to the DC to connect in the same way.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.1
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846768#comment-13846768
]
Tyler Hobbs commented on CASSANDRA-5899:
Why don't we add an {{broadcast_rpc_address}} config option? If set, that can
be used for the system.local/peers address as well as for pushed messages.
It's a simple solution with no downside (that I'm aware of), and the behavior
is familiar, since we already have something analogous with
{{broadcast_address}}.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.1
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (CASSANDRA-5899) Sends all interface in native protocol notification when rpc_address=0.0.0.0
[
https://issues.apache.org/jira/browse/CASSANDRA-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13743864#comment-13743864
]
Sylvain Lebresne commented on CASSANDRA-5899:
-
Actually, we can't easily implement that suggestion. Protocol notifications for
new nodes are not send from the new nodes themselves, so we have no way to
actually retrieve the list of all interfaces the nodes is bound to (same
problem for the peers table). That is, we would have to gossip that list, and
that's probably more than is reasonable to shove in 2.0 at this point. So well,
let me push that to 2.1 and give some more though to that.
Sends all interface in native protocol notification when rpc_address=0.0.0.0
Key: CASSANDRA-5899
URL: https://issues.apache.org/jira/browse/CASSANDRA-5899
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 2.0
For the native protocol notifications, when we send a new node notification,
we send the rpc_address of that new node. For this to be actually useful,
that address sent should be publicly accessible by the driver it is destined
to.
The problem is when rpc_address=0.0.0.0. Currently, we send the
listen_address, which is correct in the sense that we do are bind on it but
might not be accessible by client nodes.
In fact, one of the good reason to use 0.0.0.0 rpc_address would be if you
have a private network for internode communication and another for
client-server communinations, but still want to be able to issue query from
the private network for debugging. In that case, the current behavior to send
listen_address doesn't really help.
So one suggestion would be to instead send all the addresses on which the
(native protocol) server is bound to (which would still leave to the driver
the task to pick the right one, but at least it has something to pick from).
That's relatively trivial to do in practice, but it does require a minor
binary protocol break to return a list instead of just one IP, which is why
I'm tentatively marking this 2.0. Maybe we can shove that tiny change in the
final (in the protocol v2 only)? Povided we agree it's a good idea of course.
Now to be complete, for the same reasons, we would also need to store all the
addresses we are bound to in the peers table. That's also fairly simple and
the backward compatibility story is maybe a tad simpler: we could add a new
{{rpc_addresses}} column that would be a list and deprecate {{rpc_address}}
(to be removed in 2.1 for instance).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
