[ https://issues.apache.org/jira/browse/CASSANDRA-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307187#comment-17307187 ]
Adam Holmberg commented on CASSANDRA-16532: ------------------------------------------- Unfortunately I am still seeing [this|https://ci-cassandra.apache.org/job/Cassandra-trunk/362/testReport/junit/org.apache.cassandra.db/ScrubTest/testSkipScrubCorruptedCounterRowWithTool_compression/] failure when looping on this branch. With a bit of instrumentation, I can see that the scrubber is hitting this when it fails in this way: {noformat} Scrubbing BigTableReader(path='/home/vagrant/cassandra/build/test/cassandra/data/Keyspace2/Counter1-949374d48be711eb9202f742d2eeefb4/na-1-big-Data.db') (64.289KiB) WARNING: Error reading row org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: /home/vagrant/cassandra/build/test/cassandra/data/Keyspace2/Counter1-949374d48be711eb9202f742d2eeefb4/na-1-big-Data.db at org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:138) at org.apache.cassandra.db.compaction.Scrubber$RowMergingSSTableIterator.hasNext(Scrubber.java:541) at org.apache.cassandra.db.compaction.Scrubber$OrderCheckerIterator.computeNext(Scrubber.java:654) at org.apache.cassandra.db.compaction.Scrubber$OrderCheckerIterator.computeNext(Scrubber.java:574) at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) at org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74) at org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:205) at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125) at org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:149) at org.apache.cassandra.db.compaction.Scrubber.tryAppend(Scrubber.java:343) at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:233) at org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:225) at org.apache.cassandra.tools.ToolRunner.runClassAsTool(ToolRunner.java:82) at org.apache.cassandra.tools.ToolRunner$2.get(ToolRunner.java:249) at org.apache.cassandra.tools.ToolRunner$2.get(ToolRunner.java:245) at org.apache.cassandra.tools.ToolRunner.invokeSupplier(ToolRunner.java:305) at org.apache.cassandra.tools.ToolRunner.invokeClass(ToolRunner.java:253) at org.apache.cassandra.tools.ToolRunner.invokeClass(ToolRunner.java:235) at org.apache.cassandra.db.ScrubTest.testSkipScrubCorruptedCounterRowWithTool(ScrubTest.java:811) Caused by: java.io.IOException: Error building row with data deserialized from RandomAccessReader:CachingRebufferer:CompressedChunkReader.Mmap(/home/vagrant/cassandra/build/test/cassandra/data/Keyspace2/Counter1-949374d48be711eb9202f742d2eeefb4/na-1-big-Data.db - SnappyCompressor, chunk length 16384, data length 65832) at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:646) at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeOne(UnfilteredSerializer.java:487) at org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:443) at org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:84) at org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:62) at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:126) Caused by: java.lang.AssertionError at org.apache.cassandra.cache.ChunkCache$Buffer.buffer(ChunkCache.java:122) at org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:66) at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:59) at org.apache.cassandra.io.util.RebufferingInputStream.read(RebufferingInputStream.java:90) at org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68) at org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:62) at org.apache.cassandra.db.marshal.ByteArrayAccessor.read(ByteArrayAccessor.java:101) at org.apache.cassandra.db.marshal.ByteArrayAccessor.read(ByteArrayAccessor.java:38) at org.apache.cassandra.db.marshal.AbstractType.read(AbstractType.java:490) at org.apache.cassandra.db.rows.Cell$Serializer.deserialize(Cell.java:268) at org.apache.cassandra.db.rows.UnfilteredSerializer.readSimpleColumn(UnfilteredSerializer.java:655) at org.apache.cassandra.db.rows.UnfilteredSerializer.lambda$deserializeRowBody$2(UnfilteredSerializer.java:620) at org.apache.cassandra.utils.btree.BTree.applyValue(BTree.java:1294) at org.apache.cassandra.utils.btree.BTree.applyLeaf(BTree.java:1302) at org.apache.cassandra.utils.btree.BTree.apply(BTree.java:1317) at org.apache.cassandra.utils.btree.BTree.apply(BTree.java:1343) at org.apache.cassandra.db.Columns.apply(Columns.java:390) at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:612) {noformat} It only happens when compression is enabled, and when the whole test class is run (i.e.; not just one or a handful of methods). > Fix flaky testSkipScrubCorruptedCounterRowWithTool > -------------------------------------------------- > > Key: CASSANDRA-16532 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16532 > Project: Cassandra > Issue Type: Bug > Components: Test/unit > Reporter: Berenguer Blasi > Assignee: Berenguer Blasi > Priority: Normal > Fix For: 4.0-rc > > Time Spent: 20m > Remaining Estimate: 0h > > Fix flaky > [testSkipScrubCorruptedCounterRowWithTool|https://ci-cassandra.apache.org/job/Cassandra-trunk/365/testReport/junit/org.apache.cassandra.db/ScrubTest/testSkipScrubCorruptedCounterRowWithTool_compression/] -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org