[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177958#comment-15177958
]
Ariel Weisberg commented on CASSANDRA-11176:
My mistake. I thought the ticket was scoped to the test change.
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.5, 3.0.5
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
> org.apache.cassandra.utils.memory.MemtablePool$SubPool.this$0-org.apache.cassandra.utils.memory.SlabPool,
> final
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177248#comment-15177248
]
Marcus Eriksson commented on CASSANDRA-11176:
-
[~aweisberg] what do you mean? I posted the test result with that leak in the
first comment?
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.5, 3.0.5
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
>
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176638#comment-15176638
]
Ariel Weisberg commented on CASSANDRA-11176:
What about the leak that's already been detected? It's not in the new ticket
and it's kind of orthogonal.
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.5, 3.0.5
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
>
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173853#comment-15173853
]
Ariel Weisberg commented on CASSANDRA-11176:
Also I think those tests are failing because enabling the loop detection caused
it to detect loops and the test harness is seeing the errors and complaining.
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
>
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173850#comment-15173850
]
Ariel Weisberg commented on CASSANDRA-11176:
OK. Looks like it found another reference loop. The tidy can get to the tracker
through {{LogTransaction$SSTableTidier}} which references the tracker. Still
trying to figure out how to break that up.
{noformat}
ERROR [Strong-Reference-Leak-Detector:1] 2016-03-01 07:18:05,811
NoSpamLogger.java:97 - Strong self-ref loop detected
[/mnt/tmp/dtest-cjmaDm/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/ma-10-big,
private volatile java.lang.Runnable
org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.obsoletion-org.apache.cassandra.db.lifecycle.LogTransaction$SSTableTidier,
private final org.apache.cassandra.db.lifecycle.Tracker
org.apache.cassandra.db.lifecycle.LogTransaction$SSTableTidier.tracker-org.apache.cassandra.db.lifecycle.Tracker,
public final java.util.Collection
org.apache.cassandra.db.lifecycle.Tracker.subscribers-java.util.concurrent.CopyOnWriteArrayList,
public final java.util.Collection
org.apache.cassandra.db.lifecycle.Tracker.subscribers-org.apache.cassandra.db.compaction.CompactionStrategyManager,
private volatile java.util.List
org.apache.cassandra.db.compaction.CompactionStrategyManager.unrepaired-java.util.ArrayList,
transient java.lang.Object[]
java.util.ArrayList.elementData-[Ljava.lang.Object;, transient
java.lang.Object[]
java.util.ArrayList.elementData-org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,
private final java.util.Set
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.sstables-java.util.HashSet,
private transient java.util.HashMap java.util.HashSet.map-java.util.HashMap,
transient java.util.HashMap$Node[]
java.util.HashMap.table-[Ljava.util.HashMap$Node;, transient
java.util.HashMap$Node[] java.util.HashMap.table-java.util.HashMap$Node, final
java.lang.Object
java.util.HashMap$Node.key-org.apache.cassandra.io.sstable.format.big.BigTableReader,
private final
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier
org.apache.cassandra.io.sstable.format.SSTableReader.tidy-org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier,
private org.apache.cassandra.utils.concurrent.Ref
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.globalRef-org.apache.cassandra.utils.concurrent.Ref]
{noformat}
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
>
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173525#comment-15173525
]
Marcus Eriksson commented on CASSANDRA-11176:
-
I made the change to check every second and it seems it makes a bunch of tests
unrelated to this ticket fail -
http://cassci.datastax.com/job/krummas-marcuse-11176-trunk-dtest/2/
If it is OK with you, I will revert the rate-change and create a new ticket to
do the change handle the failing tests?
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
>
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172002#comment-15172002
]
Ariel Weisberg commented on CASSANDRA-11176:
Sure once a second seems reasonable. Should kick in for a lot of dtests that
way.
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
> org.apache.cassandra.utils.memory.MemtablePool$SubPool.this$0-org.apache.cassandra.utils.memory.SlabPool,
> final
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171930#comment-15171930
]
Marcus Eriksson commented on CASSANDRA-11176:
-
bq. When I tried to check the tests in cassci it looks like they didn't run?
uh, must have forgotten starting the tests, did that now
bq. I would kind of expect to have it running all the time on one core and
assume that tests have to perform acceptably in that scenario.
I just checked and during dtests both checks seem to run in <20ms, so we could
probably just change the rate (run every second maybe?)
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169885#comment-15169885
]
Ariel Weisberg commented on CASSANDRA-11176:
I looked at the changes and it looks good. Debugging the ref count by default
is a good idea, but it won't run until 15 minutes have passed. Seems like it
doesn't do much in most scenarios. I would kind of expect to have it running
all the time on one core and assume that tests have to perform acceptably in
that scenario.
When I tried to check the tests in cassci it looks like they didn't run?
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166809#comment-15166809
]
Marcus Eriksson commented on CASSANDRA-11176:
-
uh start with -Dcassandra.debugrefcount=true, run stress and wait it seems
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 3.0.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
> org.apache.cassandra.utils.memory.MemtablePool$SubPool.this$0-org.apache.cassandra.utils.memory.SlabPool,
> final
[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache
[
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166766#comment-15166766
]
Marcus Eriksson commented on CASSANDRA-11176:
-
How do I reproduce this?
> SSTableRewriter.InvalidateKeys should have a weak reference to cache
>
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 3.0.x
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references
> SSTableRewriter.InvalidateKeys which references the cache. The cache
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111
> NoSpamLogger.java:97 - Strong self-ref loop detected
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool
> org.apache.cassandra.utils.memory.MemtablePool$SubPool.this$0-org.apache.cassandra.utils.memory.SlabPool,
> final org.apache.cassandra.utils.memory.MemtableCleanerThread
>
