[
https://issues.apache.org/jira/browse/CASSANDRA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181206#comment-13181206
]
Sylvain Lebresne commented on CASSANDRA-3579:
---------------------------------------------
bq. It's slightly more intuitive for 0 gcgs to mean "this is always gc'd
immediately,"
True, though gc is more a an internal detail in that it doesn't affect whether
a column is returned by queries or not. It only affect whether it will be
removed by a compaction, but that is already so dependent of other timings than
it cannot matter. Or to put it otherwise, that one doesn't bother my own OCD.
But it doesn't really bother me either if gcBefore means before-or-equal so ...
bq. A search for < gcBefore, <= gcBefore, > gcBefore, >= gcBefore shows some of
each.
One of the goal of this patch was to actually adds consistency and I though I
had eliminated all '<=' but maybe I missed one. I forgot to search for '>' and
'>=' however. I'll fix.
bq. The isMarkedForDelete behavior you point out scares me, because that's
going to change during the two LCR passes as well. I suspect that's going to
cause subtle bugs if we don't create a fixed point in time for which we treat a
compaction as happening. (Maybe just replace controller.gcBefore with
controller.compactionTime and derive gcBefore from that as a method, instead of
a field.)
It's an option, and it was even my first idea. But it means that
isMarkedForDelete will take a timestamp, and it's called a lot in the code
base. That and the other related change, I'd be tempted to commit the patch as
is for 1.0 and maybe do the switch to 'compactionTime' for 1.1 onward.
> AssertionError in hintedhandoff - 1.0.5
> ---------------------------------------
>
> Key: CASSANDRA-3579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3579
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: RHEL 6.1 64 bit, 32 GB RAM, 8 GB allocated to JVM,
> running XFS filesystem for commit/data directories
> Reporter: Ramesh Natarajan
> Assignee: Sylvain Lebresne
> Fix For: 1.0.7
>
> Attachments: 3579.patch
>
>
> We are running a 8 node cassandra cluster running cassandra 1.0.5.
> All our CF use leveled compaction. We ran a test where we did a lot
> of inserts for 3 days. After that we started to run tests where some
> of the reads could ask for information that was inserted a while back.
> In this scenario we are seeing this assertion error in HintedHandoff.
> ERROR [HintedHandoff:3] 2011-12-05 15:42:04,324
> AbstractCassandraDaemon.java (line 133) Fatal exception in thread
> Thread[HintedHandoff:3,1,main]
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException: java.lang.AssertionError:
> originally calculated column size of 470937164 but now it is 470294247
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException: java.lang.AssertionError:
> originally calculated column size of 470937164 but now it is 470294247
> at
> org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
> at
> org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
> at
> org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: originally calculated column size of
> 470937164 but now it is 470294247
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:326)
> ... 6 more
> Caused by: java.lang.AssertionError: originally calculated column size
> of 470937164 but now it is 470294247
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
> at
> org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> ERROR [HintedHandoff:3] 2011-12-05 15:42:04,333
> AbstractCassandraDaemon.java (line 133) Fatal exception in thread
> Thread[HintedHandoff:3,1,main]
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException: java.lang.AssertionError:
> originally calculated column size of 470937164 but now it is 470294247
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException: java.lang.AssertionError:
> originally calculated column size of 470937164 but now it is 470294247
> at
> org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
> at
> org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
> at
> org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: originally calculated column size of
> 470937164 but now it is 470294247
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:326)
> ... 6 more
> Caused by: java.lang.AssertionError: originally calculated column size
> of 470937164 but now it is 470294247
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
> at
> org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> ERROR [CompactionExecutor:9931] 2011-12-05 15:42:04,333
> AbstractCassandraDaemon.java (line 133) Fatal exception in thread
> Thread[CompactionExecutor:9931,1,main]
> java.lang.AssertionError: originally calculated column size of
> 470937164 but now it is 470294247
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
> at
> org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
