In most cases, I would delete the corrupt commit log file and restart. Then run repairs on that node. I have seen cases where multiple files are corrupted and it is easier to remove all commit log files to get the node restarted.
Sean R. Durity From: Joe Obernberger <[email protected]> Sent: Friday, February 3, 2023 3:15 PM To: [email protected] Subject: [EXTERNAL] Startup fails - 4.1.0 Hi all - cluster had a power outage and one of the nodes in a 14 nodes cluster isn't starting with: DEBUG [MemtableFlushWriter: 1] 2023-02-03 13: 52: 45,468 ColumnFamilyStore. java: 1329 - Flushed to [BigTableReader(path='/data/2/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8407-big-Data. db'), INTERNAL USE Hi all - cluster had a power outage and one of the nodes in a 14 nodes cluster isn't starting with: DEBUG [MemtableFlushWriter:1] 2023-02-03 13:52:45,468 ColumnFamilyStore.java:1329 - Flushed to [BigTableReader(path='/data/2/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8407-big-Data.db'), BigTableReader(path='/data/3/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8408-big-Data.db'), BigTableReader(path='/data/4/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8409-big-Data.db'), BigTableReader(path='/data/5/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8410-big-Data.db'), BigTableReader(path='/data/6/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8411-big-Data.db'), BigTableReader(path='/data/8/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8412-big-Data.db'), BigTableReader(path='/data/9/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8413-big-Data.db')] (7 sstables, 92.858MiB), biggest 15.420MiB, smallest 10.307MiB INFO [main] 2023-02-03 13:52:45,621 CommitLogReader.java:257 - Finished reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126163.log DEBUG [main] 2023-02-03 13:52:45,622 CommitLogReader.java:266 - Reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126164.log (CL version 7, messaging version 12, compression null) INFO [main] 2023-02-03 13:52:46,811 CommitLogReader.java:257 - Finished reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126164.log DEBUG [main] 2023-02-03 13:52:46,811 CommitLogReader.java:266 - Reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126165.log (CL version 7, messaging version 12, compression null) INFO [main] 2023-02-03 13:52:47,985 CommitLogReader.java:257 - Finished reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126165.log DEBUG [main] 2023-02-03 13:52:47,986 CommitLogReader.java:266 - Reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126166.log (CL version 7, messaging version 12, compression null) INFO [main] 2023-02-03 13:52:49,282 CommitLogReader.java:257 - Finished reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126166.log DEBUG [main] 2023-02-03 13:52:49,283 CommitLogReader.java:266 - Reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126167.log (CL version 7, messaging version 12, compression null) ERROR [main] 2023-02-03 13:52:49,651 JVMStabilityInspector.java:196 - Exiting due to error while processing commit log during initialization. org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: Mutation checksum failure at 11231154 in Next section at 11230925 in CommitLog-7-1674161126167.log at org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:387) at org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:244) at org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:147) at org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:191) at org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:200) at org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:181) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:357) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:752) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:876) How to proceed? Thank you! -Joe -- This email has been checked for viruses by AVG antivirus software. https://urldefense.com/v3/__http://www.avg.com__;!!M-nmYVHPHQ!Js4daAIOfZ53leb5CNpzGIRZEtXKw9d4dmRxBCGsYe-51J4i2NkrYuazzFt8O5U-KdQ3HCo9xu4_AeqVYBNySKpz31KzFe0cOQ$<https://urldefense.com/v3/__http:/www.avg.com__;!!M-nmYVHPHQ!Js4daAIOfZ53leb5CNpzGIRZEtXKw9d4dmRxBCGsYe-51J4i2NkrYuazzFt8O5U-KdQ3HCo9xu4_AeqVYBNySKpz31KzFe0cOQ$>
