RE: Connecting to Cassandra cluster in AWS from local network
Thanks everyone for the suggestions! I have used the following code to create my cluster from my dev environment and it seems to be working perfectly: cluster = Cluster.*builder* ().addContactPoints(nodes).withAddressTranslater(*new* AddressTranslater() { *public* InetSocketAddress translate(InetSocketAddress address) { String newAddress = *null*; *if*(address != *null* && address.getAddress() != *null*) { *if*(address.getHostName().equals( "172.x.x.237")) newAddress = "54.x.x.157"; *if*(address.getHostName().equals( "172.x.x.170")) newAddress = "54.x.x.208"; *if*(address.getHostName().equals( "172.x.x.150")) newAddress = "54.x.x.142"; } *return* *new* InetSocketAddress(newAddress, address.getPort()); } }).build(); Cheers, Matt *From:* Russell Bradberry [mailto:rbradbe...@gmail.com] *Sent:* 20 April 2015 19:06 *To:* user@cassandra.apache.org *Subject:* Re: Connecting to Cassandra cluster in AWS from local network I would like to note that this will require all clients connect over the external IP address. If you have clients within Amazon that need to connect over the private IP address, this would not be possible. If you have a mix of clients that need to connect over private IP address and public, then one of the solutions outlined in https://datastax-oss.atlassian.net/browse/JAVA-145 may be more appropriate. -Russ *From: *Alex Popescu *Reply-To: * *Date: *Monday, April 20, 2015 at 2:00 PM *To: *user *Subject: *Re: Connecting to Cassandra cluster in AWS from local network You'll have to configure your nodes to: 1. use AWS internal IPs for inter-node connection (check listen_address) and 2. use the AWS public IP for client-to-node connections (check rpc_address) Depending on the setup, there might be other interesting conf options in cassandra.yaml (broadcast_address, listen_interface, rpc_interface). [1]: http://docs.datastax.com/en/cassandra/2.1/cassandra/configuration/configCassandra_yaml_r.html On Mon, Apr 20, 2015 at 9:50 AM, Jonathan Haddad wrote: Ideally you'll be on the same network, but if you can't be, you'll need to use the public ip in listen_address. On Mon, Apr 20, 2015 at 9:47 AM Matthew Johnson wrote: Hi all, I have set up a Cassandra cluster with 2.1.4 on some existing AWS boxes, just as a POC. Cassandra servers connect to each other over their internal AWS IP addresses (172.x.x.x) aliased in /etc/hosts as sales1, sales2 and sales3. I connect to it from my local dev environment using the seed’s external NAT address (54.x.x.x) aliases in my Windows hosts file as sales3 (my seed). When I try to connect, it connects fine, and can retrieve some data (I have very limited amounts of data in there, but it seems to retrieve ok), but I also get lots of stacktraces in my log where my dev environment is trying to connect to Cassandra on the internal IP (presumably the Cassandra seed node tells my dev env where to look): *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host sales3/54.x.x.142:9042 added* *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host /172.x.x.237:9042 added* *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host /172.x.x.170:9042 added* *Connected to cluster: Test Cluster* *Datatacenter: datacenter1; Host: /172.x.x.170; Rack: rack1* *Datatacenter: datacenter1; Host: sales3/54.x.x.142; Rack: rack1* *Datatacenter: datacenter1; Host: /172.x.x.237; Rack: rack1* *DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-0} Connection - Connection[sales3/54.x.x.142:9042-2, inFlight=0, closed=false] Transport initialized and ready* *DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-0} Session - Added connection pool for sales3/54.x.x.142:9042* *DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-1} Connection - Connection[/172.x.x.237:9042-1, inFlight=0, closed=false] Error connecting to /172.x.x.237:9042 (connection timed out: /172.x.x.237:9042)* *DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-1} Connection - Defuncting connection to /172.x.x.237:9042* *com.datastax.driver.core.TransportException**: [/172.x.x.237:9042] Cannot connect* Does anyone have any experience with connecting to AWS clusters from dev machines? How have you set up your aliases to get around this issue? Current setup in sales3 (seed node) cassandra.yaml: *- seeds: "sales3"* *listen_address: sales3* *rpc_address: sales3* Current setup in other nodes (eg sales2) cassandr
Re: Connecting to Cassandra cluster in AWS from local network
I would like to note that this will require all clients connect over the external IP address. If you have clients within Amazon that need to connect over the private IP address, this would not be possible. If you have a mix of clients that need to connect over private IP address and public, then one of the solutions outlined in https://datastax-oss.atlassian.net/browse/JAVA-145 may be more appropriate. -Russ From: Alex Popescu Reply-To: Date: Monday, April 20, 2015 at 2:00 PM To: user Subject: Re: Connecting to Cassandra cluster in AWS from local network You'll have to configure your nodes to: 1. use AWS internal IPs for inter-node connection (check listen_address) and 2. use the AWS public IP for client-to-node connections (check rpc_address) Depending on the setup, there might be other interesting conf options in cassandra.yaml (broadcast_address, listen_interface, rpc_interface). [1]: http://docs.datastax.com/en/cassandra/2.1/cassandra/configuration/configCassandra_yaml_r.html On Mon, Apr 20, 2015 at 9:50 AM, Jonathan Haddad wrote: Ideally you'll be on the same network, but if you can't be, you'll need to use the public ip in listen_address. On Mon, Apr 20, 2015 at 9:47 AM Matthew Johnson wrote: Hi all, I have set up a Cassandra cluster with 2.1.4 on some existing AWS boxes, just as a POC. Cassandra servers connect to each other over their internal AWS IP addresses (172.x.x.x) aliased in /etc/hosts as sales1, sales2 and sales3. I connect to it from my local dev environment using the seed’s external NAT address (54.x.x.x) aliases in my Windows hosts file as sales3 (my seed). When I try to connect, it connects fine, and can retrieve some data (I have very limited amounts of data in there, but it seems to retrieve ok), but I also get lots of stacktraces in my log where my dev environment is trying to connect to Cassandra on the internal IP (presumably the Cassandra seed node tells my dev env where to look): INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host sales3/54.x.x.142:9042 added INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host /172.x.x.237:9042 added INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host /172.x.x.170:9042 added Connected to cluster: Test Cluster Datatacenter: datacenter1; Host: /172.x.x.170; Rack: rack1 Datatacenter: datacenter1; Host: sales3/54.x.x.142; Rack: rack1 Datatacenter: datacenter1; Host: /172.x.x.237; Rack: rack1 DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-0} Connection - Connection[sales3/54.x.x.142:9042-2, inFlight=0, closed=false] Transport initialized and ready DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-0} Session - Added connection pool for sales3/54.x.x.142:9042 DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-1} Connection - Connection[/172.x.x.237:9042-1, inFlight=0, closed=false] Error connecting to /172.x.x.237:9042 (connection timed out: /172.x.x.237:9042) DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-1} Connection - Defuncting connection to /172.x.x.237:9042 com.datastax.driver.core.TransportException: [/172.x.x.237:9042] Cannot connect Does anyone have any experience with connecting to AWS clusters from dev machines? How have you set up your aliases to get around this issue? Current setup in sales3 (seed node) cassandra.yaml: - seeds: "sales3" listen_address: sales3 rpc_address: sales3 Current setup in other nodes (eg sales2) cassandra.yaml: - seeds: "sales3" listen_address: sales2 rpc_address: sales2 Thanks! Matt -- Bests, Alex Popescu | @al3xandru Sen. Product Manager @ DataStax
Re: Connecting to Cassandra cluster in AWS from local network
You'll have to configure your nodes to: 1. use AWS internal IPs for inter-node connection (check listen_address) and 2. use the AWS public IP for client-to-node connections (check rpc_address) Depending on the setup, there might be other interesting conf options in cassandra.yaml (broadcast_address, listen_interface, rpc_interface). [1]: http://docs.datastax.com/en/cassandra/2.1/cassandra/configuration/configCassandra_yaml_r.html On Mon, Apr 20, 2015 at 9:50 AM, Jonathan Haddad wrote: > Ideally you'll be on the same network, but if you can't be, you'll need to > use the public ip in listen_address. > > On Mon, Apr 20, 2015 at 9:47 AM Matthew Johnson > wrote: > >> Hi all, >> >> >> >> I have set up a Cassandra cluster with 2.1.4 on some existing AWS boxes, >> just as a POC. Cassandra servers connect to each other over their internal >> AWS IP addresses (172.x.x.x) aliased in /etc/hosts as sales1, sales2 and >> sales3. >> >> >> >> I connect to it from my local dev environment using the seed’s external >> NAT address (54.x.x.x) aliases in my Windows hosts file as sales3 (my seed). >> >> >> >> When I try to connect, it connects fine, and can retrieve some data (I >> have very limited amounts of data in there, but it seems to retrieve ok), >> but I also get lots of stacktraces in my log where my dev environment is >> trying to connect to Cassandra on the internal IP (presumably the Cassandra >> seed node tells my dev env where to look): >> >> >> >> >> >> *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New >> Cassandra host sales3/54.x.x.142:9042 added* >> >> *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New >> Cassandra host /172.x.x.237:9042 added* >> >> *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New >> Cassandra host /172.x.x.170:9042 added* >> >> *Connected to cluster: Test Cluster* >> >> *Datatacenter: datacenter1; Host: /172.x.x.170; Rack: rack1* >> >> *Datatacenter: datacenter1; Host: sales3/54.x.x.142; Rack: rack1* >> >> *Datatacenter: datacenter1; Host: /172.x.x.237; Rack: rack1* >> >> *DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver >> worker-0} Connection - Connection[sales3/54.x.x.142:9042-2, inFlight=0, >> closed=false] Transport initialized and ready* >> >> *DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver >> worker-0} Session - Added connection pool for sales3/54.x.x.142:9042* >> >> *DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver >> worker-1} Connection - Connection[/172.x.x.237:9042-1, inFlight=0, >> closed=false] Error connecting to /172.x.x.237:9042 (connection timed out: >> /172.x.x.237:9042)* >> >> *DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver >> worker-1} Connection - Defuncting connection to /172.x.x.237:9042* >> >> *com.datastax.driver.core.TransportException**: [/172.x.x.237:9042] >> Cannot connect* >> >> >> >> >> >> Does anyone have any experience with connecting to AWS clusters from dev >> machines? How have you set up your aliases to get around this issue? >> >> >> >> Current setup in sales3 (seed node) cassandra.yaml: >> >> >> >> *- seeds: "sales3"* >> >> *listen_address: sales3* >> >> *rpc_address: sales3* >> >> >> >> Current setup in other nodes (eg sales2) cassandra.yaml: >> >> >> >> *- seeds: "sales3"* >> >> *listen_address: sales2* >> >> *rpc_address: sales2* >> >> >> >> >> >> Thanks! >> >> Matt >> >> >> > -- Bests, Alex Popescu | @al3xandru Sen. Product Manager @ DataStax
Re: Connecting to Cassandra cluster in AWS from local network
There are a couple options here. You can use the built in address translator, or, write a new load balancing policy. See https://datastax-oss.atlassian.net/browse/JAVA-145 for more information. From: Jonathan Haddad Reply-To: Date: Monday, April 20, 2015 at 12:50 PM To: Subject: Re: Connecting to Cassandra cluster in AWS from local network Ideally you'll be on the same network, but if you can't be, you'll need to use the public ip in listen_address. On Mon, Apr 20, 2015 at 9:47 AM Matthew Johnson wrote: Hi all, I have set up a Cassandra cluster with 2.1.4 on some existing AWS boxes, just as a POC. Cassandra servers connect to each other over their internal AWS IP addresses (172.x.x.x) aliased in /etc/hosts as sales1, sales2 and sales3. I connect to it from my local dev environment using the seed’s external NAT address (54.x.x.x) aliases in my Windows hosts file as sales3 (my seed). When I try to connect, it connects fine, and can retrieve some data (I have very limited amounts of data in there, but it seems to retrieve ok), but I also get lots of stacktraces in my log where my dev environment is trying to connect to Cassandra on the internal IP (presumably the Cassandra seed node tells my dev env where to look): INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host sales3/54.x.x.142:9042 added INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host /172.x.x.237:9042 added INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New Cassandra host /172.x.x.170:9042 added Connected to cluster: Test Cluster Datatacenter: datacenter1; Host: /172.x.x.170; Rack: rack1 Datatacenter: datacenter1; Host: sales3/54.x.x.142; Rack: rack1 Datatacenter: datacenter1; Host: /172.x.x.237; Rack: rack1 DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-0} Connection - Connection[sales3/54.x.x.142:9042-2, inFlight=0, closed=false] Transport initialized and ready DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-0} Session - Added connection pool for sales3/54.x.x.142:9042 DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-1} Connection - Connection[/172.x.x.237:9042-1, inFlight=0, closed=false] Error connecting to /172.x.x.237:9042 (connection timed out: /172.x.x.237:9042) DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver worker-1} Connection - Defuncting connection to /172.x.x.237:9042 com.datastax.driver.core.TransportException: [/172.x.x.237:9042] Cannot connect Does anyone have any experience with connecting to AWS clusters from dev machines? How have you set up your aliases to get around this issue? Current setup in sales3 (seed node) cassandra.yaml: - seeds: "sales3" listen_address: sales3 rpc_address: sales3 Current setup in other nodes (eg sales2) cassandra.yaml: - seeds: "sales3" listen_address: sales2 rpc_address: sales2 Thanks! Matt
Re: Connecting to Cassandra cluster in AWS from local network
Ideally you'll be on the same network, but if you can't be, you'll need to use the public ip in listen_address. On Mon, Apr 20, 2015 at 9:47 AM Matthew Johnson wrote: > Hi all, > > > > I have set up a Cassandra cluster with 2.1.4 on some existing AWS boxes, > just as a POC. Cassandra servers connect to each other over their internal > AWS IP addresses (172.x.x.x) aliased in /etc/hosts as sales1, sales2 and > sales3. > > > > I connect to it from my local dev environment using the seed’s external > NAT address (54.x.x.x) aliases in my Windows hosts file as sales3 (my seed). > > > > When I try to connect, it connects fine, and can retrieve some data (I > have very limited amounts of data in there, but it seems to retrieve ok), > but I also get lots of stacktraces in my log where my dev environment is > trying to connect to Cassandra on the internal IP (presumably the Cassandra > seed node tells my dev env where to look): > > > > > > *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New > Cassandra host sales3/54.x.x.142:9042 added* > > *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New > Cassandra host /172.x.x.237:9042 added* > > *INFO 2015-04-20 16:34:14,808 [CASSANDRA-CLIENT] {main} Cluster - New > Cassandra host /172.x.x.170:9042 added* > > *Connected to cluster: Test Cluster* > > *Datatacenter: datacenter1; Host: /172.x.x.170; Rack: rack1* > > *Datatacenter: datacenter1; Host: sales3/54.x.x.142; Rack: rack1* > > *Datatacenter: datacenter1; Host: /172.x.x.237; Rack: rack1* > > *DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver > worker-0} Connection - Connection[sales3/54.x.x.142:9042-2, inFlight=0, > closed=false] Transport initialized and ready* > > *DEBUG 2015-04-20 16:34:14,901 [CASSANDRA-CLIENT] {Cassandra Java Driver > worker-0} Session - Added connection pool for sales3/54.x.x.142:9042* > > *DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver > worker-1} Connection - Connection[/172.x.x.237:9042-1, inFlight=0, > closed=false] Error connecting to /172.x.x.237:9042 (connection timed out: > /172.x.x.237:9042)* > > *DEBUG 2015-04-20 16:34:19,850 [CASSANDRA-CLIENT] {Cassandra Java Driver > worker-1} Connection - Defuncting connection to /172.x.x.237:9042* > > *com.datastax.driver.core.TransportException**: [/172.x.x.237:9042] > Cannot connect* > > > > > > Does anyone have any experience with connecting to AWS clusters from dev > machines? How have you set up your aliases to get around this issue? > > > > Current setup in sales3 (seed node) cassandra.yaml: > > > > *- seeds: "sales3"* > > *listen_address: sales3* > > *rpc_address: sales3* > > > > Current setup in other nodes (eg sales2) cassandra.yaml: > > > > *- seeds: "sales3"* > > *listen_address: sales2* > > *rpc_address: sales2* > > > > > > Thanks! > > Matt > > >