[
https://issues.apache.org/jira/browse/CASSANDRA-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115066#comment-15115066
]
Benedict commented on CASSANDRA-10697:
--------------------------------------
I've pushed a fix for this
[here|https://github.com/belliottsmith/cassandra/tree/10697-2.1]
It would be helpful if test engineering could have a look at the dtest that is
meant to check this functionality, to try and get it to exercise this problem.
I'm afraid my python is very rusty and I'm only here for a few more days.
In the process of merging this patch upwards, I found that it looks very much
like scrub is not working at all in 2.2 upwards, as the transaction we are
using doesn't get committed. So the newly scrubbed sstables should be being
deleted, and the old ones restored. Could test engineering also check why the
dtest isn't catching this problem?
> Leak detected while running offline scrub
> -----------------------------------------
>
> Key: CASSANDRA-10697
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10697
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: C* 2.1.9 on Debian Wheezy
> Reporter: mlowicki
> Assignee: Benedict
> Priority: Critical
>
> I got couple of those:
> {code}
> ERROR 05:09:15 LEAK DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3b60e162) to class
> org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@1433208674:/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-405434
> was not released before the reference was garbage collected
> {code}
> and then:
> {code}
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:99)
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81)
> at
> org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:353)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:444)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:424)
> at
> org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:378)
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:348)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:327)
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
> at
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:120)
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165)
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:192)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:158)
> at
> org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:220)
> at
> org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
