[
https://issues.apache.org/jira/browse/CASSANDRA-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-7543:
-------------------------------------------
Assignee: Yuki Morishita
> Assertion error when compacting large row with map//list field or range
> tombstone
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-7543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7543
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: linux
> Reporter: Matt Byrd
> Assignee: Yuki Morishita
> Labels: compaction, map
>
> Hi,
> So in a couple of clusters we're hitting this problem when compacting large
> rows with a schema which contains the map data-type.
> Here is an example of the error:
> {code}
> java.lang.AssertionError: incorrect row data size 87776427 written to
> /cassandra/X/Y/X-Y-tmp-ic-2381-Data.db; correct is 87845952
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:162)
> org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:163)
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
>
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:208)
> {code}
> I have a python script which reproduces the problem, by just writing lots of
> data to a single partition key with a schema that contains the map data-type.
> I added some debug logging and found that the difference in bytes seen in the
> reproduction (255) was due to the following pieces of data being written:
> {code}
> DEBUG [CompactionExecutor:3] 2014-07-13 00:38:42,891 ColumnIndex.java (line
> 168) DATASIZE writeOpenedMarker columnIndex:
> org.apache.cassandra.db.ColumnIndex$Builder@6678a9d0 firstColumn:
> [java.nio.HeapByteBuffer[pos=0 lim=34 cap=34], java.nio.HeapByteBuffer[pos=0
> lim=34 cap=34]](deletedAt=1405237116014999, localDeletion=1405237116)
> startPosition: 262476 endPosition: 262561 diff: 85
> DEBUG [CompactionExecutor:3] 2014-07-13 00:38:43,007 ColumnIndex.java (line
> 168) DATASIZE writeOpenedMarker columnIndex:
> org.apache.cassandra.db.ColumnIndex$Builder@6678a9d0 firstColumn:
> org.apache.cassandra.db.Column@3e5b5939 startPosition: 328157 endPosition:
> 328242 diff: 85
> DEBUG [CompactionExecutor:3] 2014-07-13 00:38:44,159 ColumnIndex.java (line
> 168) DATASIZE writeOpenedMarker columnIndex:
> org.apache.cassandra.db.ColumnIndex$Builder@6678a9d0 firstColumn:
> org.apache.cassandra.db.Column@fc3299b startPosition: 984105 endPosition:
> 984190 diff: 85
> {code}
> So looking at the code you can see that there are extra range tombstones
> written on the column index border (in ColumnIndex where
> tombstoneTracker.writeOpenedMarker is called) which aren't accounted for in
> LazilyCompactedRow.columnSerializedSize.
> This is where the difference comes from in the assertion error, so the
> solution is just to account for this data.
> I have a patch which does just this, by keeping track of the extra data
> written out via tombstoneTracker.writeOpenedMarker in ColumnIndex and adding
> it back to the dataSize in LazilyCompactedRow.java, where it serialises out
> the row size.
> After applying the patch the reproduction stops producing the AssertionError.
> I know this is not a problem in 2.0 + because of singe pass compaction,
> however there are lots of 1.2 clusters out there still which might run into
> this.
> Please let me know if you've any questions.
> Thanks,
> Matt
--
This message was sent by Atlassian JIRA
(v6.2#6252)
