Re: IPv6-only host, can't seem to get Cassandra to bind to a public port

2017-04-13 Thread Martijn Pieters 
On 13/04/2017, 15:06, "Khaja, Raziuddin (NIH/NLM/NCBI) [C]" 
 wrote:
> Looking at your original message: 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mail-2Darchive.com_user-40cassandra.apache.org_msg51736.html=DwIGaQ=5VD0RTtNlTh3ycd41b3MUw=xmrhtBIDZ_UcilJaq1SH8Q=pRFNVjsQoNpE4xWTky8s9kt-twcOChSSvnvSDW6jrfc=R0lHgyuqfgHkYPRDYmCpEWN1Q02x5moqZa25mRLdJ80=
>  
>  
>   I see you edited etc/cassandra/cassandra-env.sh, by changing:
>
>   
>+#JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv4Stack=true"
>
>+JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv6Addresses=true"   
>
>   First, I don’t think there is an option java.net.preferIPv6Addresses, so I 
> would recommend removing that line.

The option does exist, see 
https://docs.oracle.com/javase/8/docs/api/java/net/doc-files/net-properties.html.
 I had tried both with and without the IPv6 option.

> Second, I believe that starting in apache-cassandra-3.2, that 
> enabling/disabling the option has been moved to a file called *jvm.options* 

[snip evidence of the option moving]

> My guess right now is that you may have upgraded Cassandra from a version 
> older that 3.1 and somehow your config files are not compatible with 3.10? 

**BINGO**. Indeed, I had at some point downgraded to a different Cassandra 
version in an attempt to resolve issues with `cqlsh` (which hardcodes a CQL 
version). As a result the option was being applied **twice**, in 
/etc/cassandra/cassandra-env.sh and via /etc/cassandra/jvm.options.

Removing the switch from both locations now lets Cassandra bind to IPv6.

I now can finally drop the SSH tunnel forwarding the port in my test cluster.

Thanks!

Re: IPv6-only host, can't seem to get Cassandra to bind to a public port

2017-04-13 Thread Martijn Pieters 
From my original email: 
http://www.mail-archive.com/user@cassandra.apache.org/msg51736.html:

> My configuration changes:
>
> listen_address: 
> listen_interface_prefer_ipv6: true

listen_interface is commented out. I've just now tried again with "# 
listen_interface_prefer_ipv6: false" (option commented out), but the error 
persists. 

I've also rebooted the system, in case the upgrade from base 16.04 to 16.04.2 
left something in a funky state.

On 12/04/2017, 21:39, "Khaja, Raziuddin (NIH/NLM/NCBI) [C]" 
<raziuddin.kh...@nih.gov> wrote:

Are you specifying both the listen_address and listen_interface, or just 
one of the two?

Send,  an example of the following 3 lines.  Here is what I have on my 
2.1.16 cluster that uses ipv6:

listen_address: ::hhh::h::hhh:h
# listen_interface: eth0
# listen_interface_prefer_ipv6: false

Also, looking at my config, I can confirm that it is uneccessary or wrong 
to escape the ipv6 address with \ as I suggested before.

    -Razi
    
    On 4/12/17, 4:05 PM, "Martijn Pieters" <mjpiet...@fb.com> wrote:

From: "Khaja, Raziuddin (NIH/NLM/NCBI) [C]" <raziuddin.kh...@nih.gov>
> Maybe you have to escape the IPV6 addresses in the cassandra.yaml in 
the same way.
> I think it’s worth a try.

Nope, no luck. You get an error instead:

ERROR [main] 2017-04-12 20:03:46,899 CassandraDaemon.java:752 - 
Exception encountered during startup: Unknown listen_address 
'\:\:\:\:\:h\:hh\:h'

(actual address digits replaced with h characters).

Martijn

Re: IPv6-only host, can't seem to get Cassandra to bind to a public port

2017-04-12 Thread Martijn Pieters 
From: "Khaja, Raziuddin (NIH/NLM/NCBI) [C]" 
> Maybe you have to escape the IPV6 addresses in the cassandra.yaml in the same 
> way.
> I think it’s worth a try.

Nope, no luck. You get an error instead:

ERROR [main] 2017-04-12 20:03:46,899 CassandraDaemon.java:752 - Exception 
encountered during startup: Unknown listen_address 
'\:\:\:\:\:h\:hh\:h'
    
(actual address digits replaced with h characters).

Martijn

Re: IPv6-only host, can't seem to get Cassandra to bind to a public port

2017-04-12 Thread Martijn Pieters 
From: sai krishnam raju potturi <pskraj...@gmail.com>
> We have included the IPV6 address with scope GLOBAL, and not IPV6 with SCOPE 
> LINK in the YAML and TOPOLOGY files.  
>
> inet6 addr: 2001: *** : ** : ** : * : * :  :   Scope:Global
> inet6 addr: fe80 :: *** :  :  :  Scope:Link
>
> Not sure if this might be of relevance to the issue you are facing.

I already stated I tried both the initial email.

Martijn Pieters

Re: IPv6-only host, can't seem to get Cassandra to bind to a public port

2017-04-11 Thread Martijn Pieters 
From: sai krishnam raju potturi <pskraj...@gmail.com>
> I got a similar error, and commenting out the below line helped.
> JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv4Stack=true"
>
> Did you also include "rpc_interface_prefer_ipv6: true" in the YAML file?

No luck at all here. Yes, I had commented out that line (and also tried 
replacing it with `-Djava.net.preferIPv6Addresses=true`, included in my email. 
I also included an error to make sure it was the right file).

It all *should* work, but doesn’t. :-(

I just tried again with “rpc_interface_prefer_ipv6: true” set as well, but 
without luck. I note that I have the default “rpc_address: localhost”, so it’ll 
bind to the lo loopback, which has IPv4 configured already. Not that using 
“rpc_address: ‘::1’” instead works (same error, so I can’t bind to the IPv6 
localhost address either).

Martijn Pieters

IPv6-only host, can't seem to get Cassandra to bind to a public port

2017-04-11 Thread Martijn Pieters 
I’m having issues getting a single-node Cassandra cluster to run on a Ubuntu 
16.04 VM with only IPv6 available. I’m running Oracle Java 8 
(8u121-1~webupd8~2), Cassandra 3.10 (installed via the Cassandra 
http://www.apache.org/dist/cassandra/debian packages.)

I consistently get a “Protocol family unavailable” exception:

ERROR [main] 2017-04-11 09:54:23,991 CassandraDaemon.java:752 - Exception 
encountered during startup
java.lang.RuntimeException: java.net.SocketException: Protocol family 
unavailable
at 
org.apache.cassandra.net.MessagingService.getServerSockets(MessagingService.java:730)
 ~[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:664) 
~[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:648) 
~[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:773)
 ~[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:666) 
~[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:612) 
~[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:394) 
[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) 
[apache-cassandra-3.10.jar:3.10]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:735) 
[apache-cassandra-3.10.jar:3.10]
Caused by: java.net.SocketException: Protocol family unavailable
at sun.nio.ch.Net.bind0(Native Method) ~[na:1.8.0_121]
at sun.nio.ch.Net.bind(Net.java:433) ~[na:1.8.0_121]
at sun.nio.ch.Net.bind(Net.java:425) ~[na:1.8.0_121]
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) 
~[na:1.8.0_121]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) 
~[na:1.8.0_121]
at 
org.apache.cassandra.net.MessagingService.getServerSockets(MessagingService.java:714)
 ~[apache-cassandra-3.10.jar:3.10]
... 8 common frames omitted

`lo` (loopback) has both `inet` and `inet6` addresses, but `eth0` has no `inet` 
addresses, so only inet6 addr entries (both a local and a global scope address 
are configured).

My configuration changes:

listen_address: 
listen_interface_prefer_ipv6: true

Tracing through the source code the exception shows that it is the 
listen_address value above that throws the exception, changing it back to 
127.0.0.1 makes the server work again (but then I don’t get to use it on my 
local network). I tried both the local and the global scope IPv6 address.

I tried changing the JVM configuration to prefer IPv6 by editing 
/etc/cassandra/cassandra-env.sh:

--- etc/cassandra/cassandra-env.sh  2017-01-31 16:29:32.0 +
+++ /etc/cassandra/cassandra-env.sh 2017-04-11 09:52:51.45600 +
@@ -290,6 +290,9 @@
# to the location of the native libraries.
JVM_OPTS="$JVM_OPTS -Djava.library.path=$CASSANDRA_HOME/lib/sigar-bin"

+#JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv4Stack=true"
+JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv6Addresses=true"
+
JVM_OPTS="$JVM_OPTS $MX4J_ADDRESS"
JVM_OPTS="$JVM_OPTS $MX4J_PORT"
JVM_OPTS="$JVM_OPTS $JVM_EXTRA_OPTS"

But this makes no difference

I also tried using `listen_interface` instead, but that only changes the error 
message to:

ERROR [main] 2017-04-11 10:35:16,426 CassandraDaemon.java:752 - Exception 
encountered during startup: Configured listen_interface "eth0" could not be 
found

What else can I do?

Martijn Pieters