Jon Meredith created CASSANDRA-16068:
----------------------------------------
Summary: SSL connection to storage port when internode encryption
is disabled still try to load keystore
Key: CASSANDRA-16068
URL: https://issues.apache.org/jira/browse/CASSANDRA-16068
Project: Cassandra
Issue Type: Bug
Components: Messaging/Internode
Reporter: Jon Meredith
Starting a 4.0 cluster with internode encryption disabled can throw an
exception if the keystore is not present after CASSANDRA-15262.
Part of {{cassandra.yaml}}, the keystore is optional and it just defaults to
{{conf/.keystore}}
{code}
server_encryption_options:
internode_encryption: none
keystore: conf/.keystore
keystore_password: cassandra
truststore: conf/.truststore
truststore_password: cassandra
{code}}
Start the service and try to connect with openssl
{code}
$ openssl s_client -connect 127.0.0.1:7000
CONNECTED(00000003)
4790519404:error:140040E5:SSL routines:CONNECT_CR_SRVR_HELLO:ssl handshake
failure:ssl_pkt.c:585:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Start Time: 1597969961
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
{code}
Which triggers an ERROR message with exception
{code}
ERROR [Messaging-EventLoop-3-1] 2020-08-20 18:34:52,855
InboundConnectionInitiator.java:355 - Failed to properly handshake with peer
/127.0.0.1:61851. Closing the channel.
io.netty.handler.codec.DecoderException: java.io.IOException: failed to build
trust manager store for secure connections
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: failed to build trust manager store for secure
connections
at
org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
at
org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
at
org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
at
org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
at
org.apache.cassandra.net.InboundConnectionInitiator$OptionalSslHandler.decode(InboundConnectionInitiator.java:492)
at
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
... 17 common frames omitted
Caused by: java.nio.file.NoSuchFileException: conf/.keystore
at
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at
java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
at java.base/java.nio.file.Files.newByteChannel(Files.java:421)
at
java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
at java.base/java.nio.file.Files.newInputStream(Files.java:155)
at
org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:207)
... 23 common frames omitted
ERROR [Messaging-EventLoop-3-1] 2020-08-20 18:34:52,858
InboundConnectionInitiator.java:355 - Failed to properly handshake with peer
/127.0.0.1:61851. Closing the channel.
io.netty.handler.codec.DecoderException: java.io.IOException: failed to build
trust manager store for secure connections
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471)
at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404)
at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371)
at
io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
at
io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: failed to build trust manager store for secure
connections
at
org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
at
org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
at
org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
at
org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
at
org.apache.cassandra.net.InboundConnectionInitiator$OptionalSslHandler.decode(InboundConnectionInitiator.java:492)
at
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
... 18 common frames omitted
Caused by: java.nio.file.NoSuchFileException: conf/.keystore
at
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at
java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
at java.base/java.nio.file.Files.newByteChannel(Files.java:421)
at
java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
at java.base/java.nio.file.Files.newInputStream(Files.java:155)
at
org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:207)
... 24 common frames omitted
{code}
If the optional SSL handler is disabled, Cassandra emits this error instead.
{code}
ERROR [Messaging-EventLoop-3-1] 2020-08-20 18:44:34,338
InboundConnectionInitiator.java:358 - Failed to properly handshake with peer
/127.0.0.1:61967. Closing the channel.
io.netty.handler.codec.DecoderException:
org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read 369295616,
Expected -900387334
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471)
at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404)
at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371)
at
io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
at
io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read
369295616, Expected -900387334
at
org.apache.cassandra.net.Message.validateLegacyProtocolMagic(Message.java:333)
at
org.apache.cassandra.net.HandshakeProtocol$Initiate.maybeDecode(HandshakeProtocol.java:167)
at
org.apache.cassandra.net.InboundConnectionInitiator$Handler.initiate(InboundConnectionInitiator.java:258)
at
org.apache.cassandra.net.InboundConnectionInitiator$Handler.decode(InboundConnectionInitiator.java:251)
at
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
... 18 common frames omitted
{code}
The system should either not register the SSL handler when encryption is
disabled (the behavior before CASSANDRA-15262) or gracefully tolerate the error.
Additionally, the inbound connection should either be an INFO or a WARNING.
Outbound connection failures should be an ERROR as the endpoint is expected,
however inbound connections have to deal with anything connecting - from
security scans to malicious actors
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
