[
https://issues.apache.org/jira/browse/CASSANDRA-19590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klay updated CASSANDRA-19590:
-----------------------------
Description:
I am trying to upgrade from 2.2.19 to 3.11.17. I encountered the following
exception during the upgrade process and the 3.11.17 cannot start up.
{code:java}
ERROR [main] 2024-04-25 18:46:10,496 JVMStabilityInspector.java:124 - Exiting
due to error while processing commit log during initialization.
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
Unexpected error deserializing mutation; saved to
/tmp/mutation8318204837345269856dat. This may be caused by replaying a
mutation against a table with the same name but incompatible schema. Exception
follows: java.lang.AssertionError
at
org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:471)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:404)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:251)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:189)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:170)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:791)
{code}
h1. Reproduce
This can be reproduced deterministically by
1. Start up cassandra-2.2.19, singe node is enough (Using default configuration)
2. Execute the following commands in cqlsh
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE ks.tb (c1 INT,c2 TEXT, PRIMARY KEY (c1, c2));
ALTER TABLE ks.tb ADD c0 INT ;
INSERT INTO ks.tb (c0, c1, c2) VALUES (0,0,'RANDOM_STR');
CREATE INDEX idx ON ks.tb (c2);
ALTER TABLE ks.tb DROP c0 ;
ALTER TABLE ks.tb ADD c0 set<INT> ; {code}
3. Stop the old version.
{code:java}
bin/nodetool -h ::FFFF:127.0.0.1 flush
bin/nodetool -h ::FFFF:127.0.0.1 stopdaemon{code}
4. Copy the data and start up the new version
Upgrade crashes with the following error
{code:java}
ERROR [main] 2024-04-25 18:46:10,496 JVMStabilityInspector.java:124 - Exiting
due to error while processing commit log during initialization.
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
Unexpected error deserializing mutation; saved to
/tmp/mutation8318204837345269856dat. This may be caused by replaying a
mutation against a table with the same name but incompatible schema. Exception
follows: java.lang.AssertionError
at
org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:471)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:404)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:251)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:189)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:170)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:791){code}
I have attached the system.log when starting up the 3.11.17 node.
was:
I am trying to upgrade from 2.2.19 to 3.11.17. I encountered the following
exception during the upgrade process and the 3.11.17 cannot start up.
{code:java}
ERROR [main] 2024-04-25 18:46:10,496 JVMStabilityInspector.java:124 - Exiting
due to error while processing commit log during initialization.
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
Unexpected error deserializing mutation; saved to
/tmp/mutation8318204837345269856dat. This may be caused by replaying a
mutation against a table with the same name but incompatible schema. Exception
follows: java.lang.AssertionError
at
org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:471)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:404)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:251)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:189)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:170)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:791)
{code}
h1. Reproduce
This can be reproduced deterministically by
1. Start up cassandra-2.2.19, singe node is enough (Using default configuration)
2. Execute the following commands in cqlsh
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE ks.tb (c1 INT,c2 TEXT, PRIMARY KEY (c1, c2));
ALTER TABLE ks.tb ADD c0 INT ;
INSERT INTO ks.tb (c0, c1, c2) VALUES (0,0,'RANDOM_STR');
CREATE INDEX idx ON ks.tb (c2);
ALTER TABLE ks.tb DROP c0 ;
ALTER TABLE ks.tb ADD c0 set<INT> ; {code}
3. Stop the old version.
{code:java}
bin/nodetool -h ::FFFF:127.0.0.1 flush
bin/nodetool -h ::FFFF:127.0.0.1 stopdaemon{code}
4. Copy the data and start up the new version
Upgrade crashes with the following error
{code:java}
ERROR [main] 2024-04-25 18:46:10,496 JVMStabilityInspector.java:124 - Exiting
due to error while processing commit log during initialization.
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
Unexpected error deserializing mutation; saved to
/tmp/mutation8318204837345269856dat. This may be caused by replaying a
mutation against a table with the same name but incompatible schema. Exception
follows: java.lang.AssertionError
at
org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:471)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:404)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:251)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:189)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:170)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:791){code}
I have attached the system.log when starting up the 3.11.17 node.
> Unexpected error deserializing mutation when upgrade from 2.2.19 to 3.11.17
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-19590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19590
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
> Reporter: Klay
> Priority: Normal
> Attachments: system.log
>
>
> I am trying to upgrade from 2.2.19 to 3.11.17. I encountered the following
> exception during the upgrade process and the 3.11.17 cannot start up.
> {code:java}
> ERROR [main] 2024-04-25 18:46:10,496 JVMStabilityInspector.java:124 - Exiting
> due to error while processing commit log during initialization.
> org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
> Unexpected error deserializing mutation; saved to
> /tmp/mutation8318204837345269856dat. This may be caused by replaying a
> mutation against a table with the same name but incompatible schema.
> Exception follows: java.lang.AssertionError
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:471)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:404)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:251)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:189)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:170)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:791)
> {code}
> h1. Reproduce
> This can be reproduced deterministically by
> 1. Start up cassandra-2.2.19, singe node is enough (Using default
> configuration)
> 2. Execute the following commands in cqlsh
> {code:java}
> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c1 INT,c2 TEXT, PRIMARY KEY (c1, c2));
> ALTER TABLE ks.tb ADD c0 INT ;
> INSERT INTO ks.tb (c0, c1, c2) VALUES (0,0,'RANDOM_STR');
> CREATE INDEX idx ON ks.tb (c2);
> ALTER TABLE ks.tb DROP c0 ;
> ALTER TABLE ks.tb ADD c0 set<INT> ; {code}
> 3. Stop the old version.
> {code:java}
> bin/nodetool -h ::FFFF:127.0.0.1 flush
> bin/nodetool -h ::FFFF:127.0.0.1 stopdaemon{code}
> 4. Copy the data and start up the new version
> Upgrade crashes with the following error
> {code:java}
> ERROR [main] 2024-04-25 18:46:10,496 JVMStabilityInspector.java:124 - Exiting
> due to error while processing commit log during initialization.
> org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
> Unexpected error deserializing mutation; saved to
> /tmp/mutation8318204837345269856dat. This may be caused by replaying a
> mutation against a table with the same name but incompatible schema.
> Exception follows: java.lang.AssertionError
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:471)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:404)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:251)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:189)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:170)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:791){code}
> I have attached the system.log when starting up the 3.11.17 node.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
