[
https://issues.apache.org/jira/browse/CASSANDRA-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15138560#comment-15138560
]
Marcus Eriksson commented on CASSANDRA-10697:
---------------------------------------------
fix against 2.1 here:
https://github.com/krummas/cassandra/commits/marcuse/10697 - benedicts fix
above was not enough to get rid of the error messages, we also need to release
the self ref when doing offline scrubbing
dtest which reproduces the error here:
https://github.com/krummas/cassandra-dtest/commits/marcuse/10697
test runs:
http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10697-dtest/
http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10697-testall
bq. I found that it looks very much like scrub is not working at all in 2.2
upwards
I'll investigate this before merging to 2.2+
> Leak detected while running offline scrub
> -----------------------------------------
>
> Key: CASSANDRA-10697
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10697
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: C* 2.1.9 on Debian Wheezy
> Reporter: mlowicki
> Assignee: Marcus Eriksson
> Priority: Critical
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> I got couple of those:
> {code}
> ERROR 05:09:15 LEAK DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3b60e162) to class
> org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@1433208674:/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-405434
> was not released before the reference was garbage collected
> {code}
> and then:
> {code}
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:99)
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81)
> at
> org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:353)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:444)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:424)
> at
> org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:378)
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:348)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:327)
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
> at
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:120)
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165)
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:192)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:158)
> at
> org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:220)
> at
> org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
