[
https://issues.apache.org/jira/browse/CASSANDRA-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710858#comment-14710858
]
Stefania commented on CASSANDRA-10159:
--------------------------------------
bq. If we fail to complete a transaction, we should just immediately delete the
log file, no?
I meant on startup when we remove the leftovers, sorry.
bq. (Or leave it in a state where we can do so on restart)
That's what we aim to achieve but if there is a bug or manual intervention we
may not be able to.
We can change the log message, "Possible disk corruption" is an attempt to draw
the attention of the operators, any preferences?
> Incorrect last update time causes dtest to fail due to unexpected errors
> ------------------------------------------------------------------------
>
> Key: CASSANDRA-10159
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10159
> Project: Cassandra
> Issue Type: Bug
> Reporter: Stefania
> Assignee: Stefania
> Fix For: 3.0.0 rc1
>
> Attachments: node2.log
>
>
> Some dtests are failing as follows:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/96/testReport/counter_tests/TestCounters/upgrade_test/
> {code}
> Unexpected error in node2 node log: ['ERROR [main] 2015-08-23 11:25:52,701
> TransactionLog.java:246 - Possible disk corruption detected for sstable
> [ma-2-big], record [REMOVE:[ma-2-big,1440329048000,8]]: last update time [Thu
> Jan 01 00:00:00 UTC 1970] should have been [Sun Aug 23 11:24:08 UTC 2015]
> ERROR [main] 2015-08-23 11:25:52,709 TransactionLog.java:992 - Possible disk
> corruption: failed to read transaction log
> /mnt/tmp/dtest-E0OvQC/test/node2/data/system/local-7ad54392bcdd35a684174e047860b377/ma_txn_compaction_90eda9f0-4989-11e5-86bd-f32569933441.log
>
> org.apache.cassandra.db.lifecycle.TransactionLog$CorruptTransactionLogException:
> Failed to verify transaction 90eda9f0-4989-11e5-86bd-f32569933441 record
> [REMOVE:[ma-2-big,1440329048000,8]]: possible disk corruption, aborting \tat
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile.readRecords(TransactionLog.java:349)
> ~[main/:na] \tat
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionData.readLogFile(TransactionLog.java:574)
> ~[main/:na] \tat
> org.apache.cassandra.db.lifecycle.TransactionLog.removeUnfinishedLeftovers(TransactionLog.java:988)
> ~[main/:na] \tat
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.removeUnfinishedLeftovers(LifecycleTransaction.java:548)
> [main/:na] \tat
> org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:584)
> [main/:na] \tat
> org.apache.cassandra.service.StartupChecks$7.execute(StartupChecks.java:274)
> [main/:na] \tat
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:103)
> [main/:na] \tat
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:166)
> [main/:na] \tat
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516)
> [main/:na] \tat
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622)
> [main/:na] ERROR [main] 2015-08-23 11:25:52,710 TransactionLog.java:998 -
> Failed to remove unfinished transaction leftovers
> org.apache.cassandra.db.lifecycle.TransactionLog$CorruptTransactionLogException:
> Failed to verify transaction 90eda9f0-4989-11e5-86bd-f32569933441 record
> [REMOVE:[ma-2-big,1440329048000,8]]: possible disk corruption, aborting \tat
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile.readRecords(TransactionLog.java:349)
> ~[main/:na] \tat
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionData.readLogFile(TransactionLog.java:574)
> ~[main/:na] \tat
> org.apache.cassandra.db.lifecycle.TransactionLog.removeUnfinishedLeftovers(TransactionLog.java:988)
> ~[main/:na] \tat
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.removeUnfinishedLeftovers(LifecycleTransaction.java:548)
> [main/:na] \tat
> org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:584)
> [main/:na] \tat
> org.apache.cassandra.service.StartupChecks$7.execute(StartupChecks.java:274)
> [main/:na] \tat
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:103)
> [main/:na] \tat
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:166)
> [main/:na] \tat
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516)
> [main/:na] \tat
> {code}
> My best guess is that before reading the update time we should check that the
> file actually exists.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
