Re: nodetool breaks on firewall ?

2014-12-13 Thread Kevin Burton 
I ended up working around this by allowing the host to connect to its own
fronted port.

Figured it’s a reasonable solution.

On Fri, Dec 12, 2014 at 12:38 PM, Ryan Svihla rsvi...@datastax.com wrote:

 well did you restart cassandra after changing the JVM_OPTS to match your
 desired address?

 On Fri, Dec 12, 2014 at 2:34 PM, Kevin Burton bur...@spinn3r.com wrote:

 Oh.  and if I specify —host it still doesn’t work. Very weird.

 On Fri, Dec 12, 2014 at 12:33 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 OK..I’m stracing it and it’s definitely trying to connect to 173… here’s
 the log line below.  (anonymized).

 the question is why.. is cassandra configured to return something on the
 public address via JMX? I guess I could dump all of JMX metrics and figure
 it out.

 [pid 32331] connect(41, {sa_family=AF_INET6, sin6_port=htons(7199),
 inet_pton(AF_INET6, :::173.x.x.x, sin6_addr), sin6_flowinfo=0,
 sin6_scope_id=0}, 28 unfinished ...

 On Fri, Dec 12, 2014 at 12:20 PM, Ryan Svihla rsvi...@datastax.com
 wrote:

 is appears to be localhost, I imagine the issue is more you changed the
 rpc_address to not be localhost anymore


 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java

 lines 87 and 88
 private static final String DEFAULT_HOST = 127.0.0.1;
 private static final int DEFAULT_PORT = 7199;

 On Fri, Dec 12, 2014 at 2:09 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
 determine the host to read from by default?

 The problem is that somehow it wants to read from the public interface
 (which is fire walled)

 On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com
 wrote:

 yes the node needs to restart to have cassandra-env.sh take effect,
 and the links you're providing are about making cassandra's JMX bind to 
 the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something 
 like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 I have a firewall I need to bring up to keep our boxes off the
 Internet (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image:
 linkedin.png] http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database
 technology, delivering Apache Cassandra to the world’s most innovative
 enterprises. Datastax is built to be agile, always-on, and predictably
 scalable to any size. With more than 500 customers in 45 countries, 
 DataStax
 is the database technology and transactional backbone of choice for the
 worlds most innovative companies such as Netflix, Adobe, Intuit, and 
 eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image:
 linkedin.png] http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog:

Re: nodetool breaks on firewall ?

2014-12-12 Thread Ryan Svihla 
yes the node needs to restart to have cassandra-env.sh take effect, and the
links you're providing are about making cassandra's JMX bind to the
interface you want, so nodetool isn't really the issue, nodetool can just
take an ip argument to connect to the interface you desire.Something like:

nodetool status -h 10.1.1.100



On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com wrote:

 I have a firewall I need to bring up to keep our boxes off the Internet
 (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



-- 

[image: datastax_logo.png] http://www.datastax.com/

Ryan Svihla

Solution Architect

[image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
http://www.linkedin.com/pub/ryan-svihla/12/621/727/

DataStax is the fastest, most scalable distributed database technology,
delivering Apache Cassandra to the world’s most innovative enterprises.
Datastax is built to be agile, always-on, and predictably scalable to any
size. With more than 500 customers in 45 countries, DataStax is the
database technology and transactional backbone of choice for the worlds
most innovative companies such as Netflix, Adobe, Intuit, and eBay.

Re: nodetool breaks on firewall ?

2014-12-12 Thread Kevin Burton 
AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
determine the host to read from by default?

The problem is that somehow it wants to read from the public interface
(which is fire walled)

On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com wrote:

 yes the node needs to restart to have cassandra-env.sh take effect, and
 the links you're providing are about making cassandra's JMX bind to the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com wrote:

 I have a firewall I need to bring up to keep our boxes off the Internet
 (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com

Re: nodetool breaks on firewall ?

2014-12-12 Thread Ryan Svihla 
is appears to be localhost, I imagine the issue is more you changed the
rpc_address to not be localhost anymore

https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java

lines 87 and 88
private static final String DEFAULT_HOST = 127.0.0.1;
private static final int DEFAULT_PORT = 7199;

On Fri, Dec 12, 2014 at 2:09 PM, Kevin Burton bur...@spinn3r.com wrote:

 AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
 determine the host to read from by default?

 The problem is that somehow it wants to read from the public interface
 (which is fire walled)

 On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com wrote:

 yes the node needs to restart to have cassandra-env.sh take effect, and
 the links you're providing are about making cassandra's JMX bind to the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com wrote:

 I have a firewall I need to bring up to keep our boxes off the Internet
 (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



-- 

[image: datastax_logo.png] http://www.datastax.com/

Ryan Svihla

Solution Architect

[image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
http://www.linkedin.com/pub/ryan-svihla/12/621/727/

DataStax is the fastest, most scalable distributed database technology,
delivering Apache Cassandra to the world’s most innovative enterprises.
Datastax is built to be agile, always-on, and predictably scalable to any
size. With more than 500 customers in 45 countries, DataStax is the
database technology and transactional backbone of choice for the worlds
most innovative companies such as Netflix, Adobe, Intuit, and eBay.

Re: nodetool breaks on firewall ?

2014-12-12 Thread Ryan Svihla 
hmm I was hoping it was changed in 2.1

https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/tools/NodeTool.java

but still localhost, sorry I can't tell you why it would go to the public
interface..maybe someone added a shell alias?

On Fri, Dec 12, 2014 at 2:20 PM, Ryan Svihla rsvi...@datastax.com wrote:

 is appears to be localhost, I imagine the issue is more you changed the
 rpc_address to not be localhost anymore


 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java

 lines 87 and 88
 private static final String DEFAULT_HOST = 127.0.0.1;
 private static final int DEFAULT_PORT = 7199;

 On Fri, Dec 12, 2014 at 2:09 PM, Kevin Burton bur...@spinn3r.com wrote:

 AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
 determine the host to read from by default?

 The problem is that somehow it wants to read from the public interface
 (which is fire walled)

 On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com
 wrote:

 yes the node needs to restart to have cassandra-env.sh take effect, and
 the links you're providing are about making cassandra's JMX bind to the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 I have a firewall I need to bring up to keep our boxes off the Internet
 (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.



-- 

[image: datastax_logo.png] http://www.datastax.com/

Ryan Svihla

Solution Architect

[image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
http://www.linkedin.com/pub/ryan-svihla/12/621/727/

DataStax is the fastest, most scalable distributed database technology,
delivering Apache Cassandra to the world’s most innovative enterprises.
Datastax is built to be agile, always-on, and predictably scalable to any
size. With more than 500 customers in 45 countries, DataStax is the
database technology and transactional backbone of choice for the worlds
most innovative companies such as Netflix, Adobe, Intuit, and eBay.

Re: nodetool breaks on firewall ?

2014-12-12 Thread Kevin Burton 
Oh.  and if I specify —host it still doesn’t work. Very weird.

On Fri, Dec 12, 2014 at 12:33 PM, Kevin Burton bur...@spinn3r.com wrote:

 OK..I’m stracing it and it’s definitely trying to connect to 173… here’s
 the log line below.  (anonymized).

 the question is why.. is cassandra configured to return something on the
 public address via JMX? I guess I could dump all of JMX metrics and figure
 it out.

 [pid 32331] connect(41, {sa_family=AF_INET6, sin6_port=htons(7199),
 inet_pton(AF_INET6, :::173.x.x.x, sin6_addr), sin6_flowinfo=0,
 sin6_scope_id=0}, 28 unfinished ...

 On Fri, Dec 12, 2014 at 12:20 PM, Ryan Svihla rsvi...@datastax.com
 wrote:

 is appears to be localhost, I imagine the issue is more you changed the
 rpc_address to not be localhost anymore


 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java

 lines 87 and 88
 private static final String DEFAULT_HOST = 127.0.0.1;
 private static final int DEFAULT_PORT = 7199;

 On Fri, Dec 12, 2014 at 2:09 PM, Kevin Burton bur...@spinn3r.com wrote:

 AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
 determine the host to read from by default?

 The problem is that somehow it wants to read from the public interface
 (which is fire walled)

 On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com
 wrote:

 yes the node needs to restart to have cassandra-env.sh take effect, and
 the links you're providing are about making cassandra's JMX bind to the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 I have a firewall I need to bring up to keep our boxes off the
 Internet (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image:
 linkedin.png] http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com

Re: nodetool breaks on firewall ?

2014-12-12 Thread Kevin Burton 
OK..I’m stracing it and it’s definitely trying to connect to 173… here’s
the log line below.  (anonymized).

the question is why.. is cassandra configured to return something on the
public address via JMX? I guess I could dump all of JMX metrics and figure
it out.

[pid 32331] connect(41, {sa_family=AF_INET6, sin6_port=htons(7199),
inet_pton(AF_INET6, :::173.x.x.x, sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, 28 unfinished ...

On Fri, Dec 12, 2014 at 12:20 PM, Ryan Svihla rsvi...@datastax.com wrote:

 is appears to be localhost, I imagine the issue is more you changed the
 rpc_address to not be localhost anymore


 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java

 lines 87 and 88
 private static final String DEFAULT_HOST = 127.0.0.1;
 private static final int DEFAULT_PORT = 7199;

 On Fri, Dec 12, 2014 at 2:09 PM, Kevin Burton bur...@spinn3r.com wrote:

 AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
 determine the host to read from by default?

 The problem is that somehow it wants to read from the public interface
 (which is fire walled)

 On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com
 wrote:

 yes the node needs to restart to have cassandra-env.sh take effect, and
 the links you're providing are about making cassandra's JMX bind to the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 I have a firewall I need to bring up to keep our boxes off the Internet
 (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com

Re: nodetool breaks on firewall ?

2014-12-12 Thread Ryan Svihla 
well did you restart cassandra after changing the JVM_OPTS to match your
desired address?

On Fri, Dec 12, 2014 at 2:34 PM, Kevin Burton bur...@spinn3r.com wrote:

 Oh.  and if I specify —host it still doesn’t work. Very weird.

 On Fri, Dec 12, 2014 at 12:33 PM, Kevin Burton bur...@spinn3r.com wrote:

 OK..I’m stracing it and it’s definitely trying to connect to 173… here’s
 the log line below.  (anonymized).

 the question is why.. is cassandra configured to return something on the
 public address via JMX? I guess I could dump all of JMX metrics and figure
 it out.

 [pid 32331] connect(41, {sa_family=AF_INET6, sin6_port=htons(7199),
 inet_pton(AF_INET6, :::173.x.x.x, sin6_addr), sin6_flowinfo=0,
 sin6_scope_id=0}, 28 unfinished ...

 On Fri, Dec 12, 2014 at 12:20 PM, Ryan Svihla rsvi...@datastax.com
 wrote:

 is appears to be localhost, I imagine the issue is more you changed the
 rpc_address to not be localhost anymore


 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java

 lines 87 and 88
 private static final String DEFAULT_HOST = 127.0.0.1;
 private static final int DEFAULT_PORT = 7199;

 On Fri, Dec 12, 2014 at 2:09 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 AH! … ok. I didn’t see that nodetool took a host.  Hm.. How does it
 determine the host to read from by default?

 The problem is that somehow it wants to read from the public interface
 (which is fire walled)

 On Fri, Dec 12, 2014 at 5:19 AM, Ryan Svihla rsvi...@datastax.com
 wrote:

 yes the node needs to restart to have cassandra-env.sh take effect,
 and the links you're providing are about making cassandra's JMX bind to 
 the
 interface you want, so nodetool isn't really the issue, nodetool can just
 take an ip argument to connect to the interface you desire.Something like:

 nodetool status -h 10.1.1.100



 On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton bur...@spinn3r.com
 wrote:

 I have a firewall I need to bring up to keep our boxes off the
 Internet (obviously).

 The problem is that once I do nodetool doesn’t work anymore.

 There’s a bunch of advice on this on the Internet:


 http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem

 http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

 .. almost all the advice talks about editing cassandra-env.sh

 The problem here is that nodetool doesn’t use the JVM_OPTS param so
 anything added there isn’t used by nodetool.  (at least in 2.0.9)

 I want to force cassandra to always use our 10x network.

 Any advice here?  Do I have to do a forced cassandra restart for my
 cassandra-env.sh to take effect?



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image:
 linkedin.png] http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database
 technology, delivering Apache Cassandra to the world’s most innovative
 enterprises. Datastax is built to be agile, always-on, and predictably
 scalable to any size. With more than 500 customers in 45 countries, 
 DataStax
 is the database technology and transactional backbone of choice for the
 worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --

 [image: datastax_logo.png] http://www.datastax.com/

 Ryan Svihla

 Solution Architect

 [image: twitter.png] https://twitter.com/foundev [image: linkedin.png]
 http://www.linkedin.com/pub/ryan-svihla/12/621/727/

 DataStax is the fastest, most scalable distributed database technology,
 delivering Apache Cassandra to the world’s most innovative enterprises.
 Datastax is built to be agile, always-on, and predictably scalable to any
 size. With more than 500 customers in 45 countries, DataStax is the
 database technology and transactional backbone of choice for the worlds
 most innovative companies such as Netflix, Adobe, Intuit, and eBay.




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



-- 

[image: datastax_logo.png] http://www.datastax.com/

Ryan Svihla

Solution Architect

nodetool breaks on firewall ?

2014-12-11 Thread Kevin Burton 
I have a firewall I need to bring up to keep our boxes off the Internet
(obviously).

The problem is that once I do nodetool doesn’t work anymore.

There’s a bunch of advice on this on the Internet:

http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-trying-to-connect-to-rem
http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootConnectionsFail_r.html

.. almost all the advice talks about editing cassandra-env.sh

The problem here is that nodetool doesn’t use the JVM_OPTS param so
anything added there isn’t used by nodetool.  (at least in 2.0.9)

I want to force cassandra to always use our 10x network.

Any advice here?  Do I have to do a forced cassandra restart for my
cassandra-env.sh to take effect?



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com