[
https://issues.apache.org/jira/browse/CASSANDRA-17367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489462#comment-17489462
]
Dmitry Potepalov edited comment on CASSANDRA-17367 at 2/9/22, 10:59 AM:
------------------------------------------------------------------------
Patches attached to the ticket. With this change, sstableloader performs the
handshake and my migration succeeded.
Just in case - sstableloader command I used to reproduce the issue on 4.0.0:
{{sstableloader -f cassandra.yaml -d <ip>:<cqlsh port> -ssp <legacy ssl port on
target> -u user -pw password <path to sstables>}}
and the exception in sstableloader output:
{noformat}
15:38:00.725 [main] DEBUG
org.apache.cassandra.streaming.async.NettyStreamingMessageSender - [Stream
#16fda750-88f5-11ec-b4ef-0bc683c3f5ea] Closing stream connection channels on
/35.205.83.18:7000
15:38:00.728 [main] ERROR org.apache.cassandra.streaming.StreamSession -
[Stream #16fda750-88f5-11ec-b4ef-0bc683c3f5ea] Streaming error occurred on
session with peer 35.205.83.18:7000
java.io.IOException: failed to connect to /35.205.83.18:7000 for streaming data
at
org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:55)
at
org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:52)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:199)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:180)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:245)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:149)
at
org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:362)
at
org.apache.cassandra.streaming.StreamCoordinator.startSession(StreamCoordinator.java:262)
at
org.apache.cassandra.streaming.StreamCoordinator.access$700(StreamCoordinator.java:36)
at
org.apache.cassandra.streaming.StreamCoordinator$HostStreamingData.connectAllStreamSessions(StreamCoordinator.java:308)
at
org.apache.cassandra.streaming.StreamCoordinator.connectAllStreamSessions(StreamCoordinator.java:107)
at
org.apache.cassandra.streaming.StreamCoordinator.connect(StreamCoordinator.java:101)
at
org.apache.cassandra.streaming.StreamResultFuture.createInitiator(StreamResultFuture.java:98)
at
org.apache.cassandra.streaming.StreamPlan.execute(StreamPlan.java:179)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:215)
at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:83)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:51)
Caused by: java.nio.channels.ClosedChannelException: null
at
org.apache.cassandra.net.OutboundConnectionInitiator$Handler.channelInactive(OutboundConnectionInitiator.java:248)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
at
io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{noformat}
was (Author: JIRAUSER284878):
Patches attached to the ticket. With this change, sstableloader performs the
handshake and my migration succeeded.
Just in case - sstableloader command I used to reproduce the issue on 4.0.0:
{{sstableloader -f cassandra.yaml -d <ip>:<cqlsh port> -ssp <legacy ssl port on
target> -u user -pw password <path to sstables>}}
and the exception in sstableloader output:
{noformat}
15:38:00.725 [main] DEBUG
org.apache.cassandra.streaming.async.NettyStreamingMessageSender - [Stream
#16fda750-88f5-11ec-b4ef-0bc683c3f5ea] Closing stream connection channels on
/35.205.83.18:7000 15:38:00.728 [main] ERROR
org.apache.cassandra.streaming.StreamSession - [Stream
#16fda750-88f5-11ec-b4ef-0bc683c3f5ea] Streaming error occurred on session with
peer 35.205.83.18:7000 java.io.IOException: failed to connect to
/35.205.83.18:7000 for streaming data at
org.apache.cassandra.streaming.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:55)
at
org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:52)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.createChannel(NettyStreamingMessageSender.java:199)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.setupControlMessageChannel(NettyStreamingMessageSender.java:180)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.sendMessage(NettyStreamingMessageSender.java:245)
at
org.apache.cassandra.streaming.async.NettyStreamingMessageSender.initialize(NettyStreamingMessageSender.java:149)
at org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:362)
at
org.apache.cassandra.streaming.StreamCoordinator.startSession(StreamCoordinator.java:262)
at
org.apache.cassandra.streaming.StreamCoordinator.access$700(StreamCoordinator.java:36)
at
org.apache.cassandra.streaming.StreamCoordinator$HostStreamingData.connectAllStreamSessions(StreamCoordinator.java:308)
at
org.apache.cassandra.streaming.StreamCoordinator.connectAllStreamSessions(StreamCoordinator.java:107)
at
org.apache.cassandra.streaming.StreamCoordinator.connect(StreamCoordinator.java:101)
at
org.apache.cassandra.streaming.StreamResultFuture.createInitiator(StreamResultFuture.java:98)
at org.apache.cassandra.streaming.StreamPlan.execute(StreamPlan.java:179) at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:215) at
org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:83) at
org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:51) Caused by:
java.nio.channels.ClosedChannelException: null at
org.apache.cassandra.net.OutboundConnectionInitiator$Handler.channelInactive(OutboundConnectionInitiator.java:248)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
at
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
at
io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829) {noformat}
> sstableloader ignores streaming encryption settings
> ---------------------------------------------------
>
> Key: CASSANDRA-17367
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17367
> Project: Cassandra
> Issue Type: Bug
> Reporter: Dmitry Potepalov
> Priority: Normal
> Attachments: 17367-4.0.txt, 17367-trunk.txt
>
>
> Reproducible in Cassandra 4.x. If one configures encryption for streaming in
> config yaml fed to sstableloader like this
> {{server_encryption_options:}}
> {{ internode_encryption: all}}
> {{ keystore: sstableloader.keystore.p12}}
> {{ keystore_password: changeit}}
> {{ truststore: sstableloader.truststore.jks}}
> {{ truststore_password: changeit}}
> then sstableloader should perform an SSL handshake on the streaming
> connections and encrypt the payload. But this does not happen. Judging by the
> TCPdump of the outgoing traffic on the internode port, sstableloader sends
> plaintext traffic. This is the TCP payload of the first packet that
> sstableloader sends after establishing TCP connection:
> {{ca 55 2d fa 0c 0c 0c 08 06 0a f0 01 f9 1b 58 a8 32 f2 d0}}
> The first 4 bytes look like Cassandra protocol magic, not like a client hello.
> I've discovered the issue while trying to migrate some data to a Cassandra 4
> listening on the legacy ssl storage port (therefore, accepting only encrypted
> connections on that port). Streaming phase of the migration failed with a
> "connection closed" error, which hints that the connection was closed
> server-side.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
