[
https://issues.apache.org/jira/browse/CASSANDRA-18595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733377#comment-17733377
]
Sam Tunnicliffe commented on CASSANDRA-18595:
---------------------------------------------
I imagine it's probably rather annoying to fix, but anyway I definitely agree
with Brandon. In the current TCM branch we actually forbid schema alteration
while topology changes are in progress. In an upcoming patch (coming soon, I
promise), we narrow that down to only alterations which affect which
replication, but we could also prevent tables being dropped while movements are
happening.
> Dropped table causes node to get stuck in JOINING
> -------------------------------------------------
>
> Key: CASSANDRA-18595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18595
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Cameron Zemek
> Priority: Normal
> Attachments: cass_stream.patch
>
>
> To facilitate reproducibility of this issue I have created a patch.
> [^cass_stream.patch]
> It can also be reproduced without patch by dropping table while node is doing
> stream for join.
> {code:java}
> cd ~/dev/cassandra
> git checkout 3.11.14
> git apply ~/downloads/cass_stream.patch
> ant clean
> ant
> ccm create -n 3 test
> ccm node1 start
> ccm node2 start
> ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = {'class':
> 'NetworkTopologyStrategy', 'datacenter1': '3'};"
> # receive warning since nodes < RF
> ccm node1 cqlsh -e "CREATE TABLE test.streamtest(stream_id int PRIMARY KEY,
> payload TEXT);"
> ccm node1 cqlsh -e "INSERT INTO test.streamtest(stream_id, payload) VALUES(1,
> 'hello world')"
> ccm flush
> sed -i 's/auto_bootstrap: false/auto_bootstrap: true/'
> ~/.ccm/test/node3/node.conf
> sed -i 's/auto_bootstrap: false/auto_bootstrap: true/'
> ~/.ccm/test/node3/conf/cassandra.yaml
> sed -i 's/seeds: 127.0.0.1,127.0.0.2,127.0.0.3/seeds: 127.0.0.1,127.0.0.2/'
> ~/.ccm/test/node3/conf/cassandra.yaml
> touch ~/.ccm/test/node3/logs/system.log
> ccm node3 start
> tail -f ~/.ccm/test/node3/logs/system.log{code}
> The logs will display something like:
> {noformat}
> ERROR [STREAM-IN-/127.0.0.1:7000] 2023-06-14 11:08:37,399
> StreamSession.java:609 - [Stream #fdd71710-0a4f-11ee-a768-c3d20ff8c2d1]
> Streaming error occurred on session with peer 127.0.0.1
> java.io.IOException: streamtest 38f79d70-0a4f-11ee-9849-a999e71c8739 CF was
> dropped during compressed streaming
> ERROR [main] 2023-06-14 11:08:37,405 StorageService.java:1630 - Error while
> waiting on bootstrap to complete. Bootstrap will have to be
> restarted.{noformat}
> The problem is in this use case the customer is rotating out tables every few
> hours and so resuming the bootstrap keeps failing. In my opinion this is a
> bug and Cassandra should be robust enough to continue streaming if a table is
> dropped.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
