[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-13432:
-
Resolution: Won't Fix
Status: Resolved (was: Open)
It is now far too late in 2.x's life for this.
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
>Priority: Normal
> Fix For: 2.1.x
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-13432:
-
Status: Open (was: Patch Available)
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
>Priority: Normal
> Fix For: 2.1.x
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-13432:
-
Component/s: Core
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
>Priority: Major
> Fix For: 2.1.x
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
> at
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Corentin Chary updated CASSANDRA-13432:
---
Attachment: (was: CASSANDRA-13432.patch)
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
> Fix For: 2.1.x
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
> at
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Corentin Chary updated CASSANDRA-13432:
---
Status: Patch Available (was: Open)
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-13432.patch
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
> at
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Corentin Chary updated CASSANDRA-13432:
---
Attachment: CASSANDRA-13432.patch
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-13432.patch
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
> at
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-13432:
-
Environment:
cassandra 2.1.15
> MemtableReclaimMemory can get stuck because of lack of timeout in
> getTopLevelColumns()
> --
>
> Key: CASSANDRA-13432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13432
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra 2.1.15
>Reporter: Corentin Chary
> Fix For: 2.1.x
>
>
> This might affect 3.x too, I'm not sure.
> {code}
> $ nodetool tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 32135875 0
> 0
> ReadStage 114 0 29492940 0
> 0
> RequestResponseStage 0 0 86090931 0
> 0
> ReadRepairStage 0 0 166645 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> MiscStage 0 0 0 0
> 0
> HintedHandoff 0 0 47 0
> 0
> GossipStage 0 0 188769 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> CommitLogArchiver 0 0 0 0
> 0
> CompactionExecutor0 0 86835 0
> 0
> ValidationExecutor0 0 0 0
> 0
> MigrationStage0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> PendingRangeCalculator0 0 92 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0563 0
> 0
> MemtablePostFlush 0 0 1500 0
> 0
> MemtableReclaimMemory 129534 0
> 0
> Native-Transport-Requests41 0 54819182 0
> 1896
> {code}
> {code}
> "MemtableReclaimMemory:195" - Thread t@6268
>java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
> at
> org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "SharedPool-Worker-195" - Thread t@989
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
> at
> org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
> at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
> at
> org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
> at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:156)
>
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Corentin Chary updated CASSANDRA-13432:
---
Description:
This might affect 3.x too, I'm not sure.
{code}
$ nodetool tpstats
Pool NameActive Pending Completed Blocked All
time blocked
MutationStage 0 0 32135875 0
0
ReadStage 114 0 29492940 0
0
RequestResponseStage 0 0 86090931 0
0
ReadRepairStage 0 0 166645 0
0
CounterMutationStage 0 0 0 0
0
MiscStage 0 0 0 0
0
HintedHandoff 0 0 47 0
0
GossipStage 0 0 188769 0
0
CacheCleanupExecutor 0 0 0 0
0
InternalResponseStage 0 0 0 0
0
CommitLogArchiver 0 0 0 0
0
CompactionExecutor0 0 86835 0
0
ValidationExecutor0 0 0 0
0
MigrationStage0 0 0 0
0
AntiEntropyStage 0 0 0 0
0
PendingRangeCalculator0 0 92 0
0
Sampler 0 0 0 0
0
MemtableFlushWriter 0 0563 0
0
MemtablePostFlush 0 0 1500 0
0
MemtableReclaimMemory 129534 0
0
Native-Transport-Requests41 0 54819182 0
1896
{code}
{code}
"MemtableReclaimMemory:195" - Thread t@6268
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
at
org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
at
org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"SharedPool-Worker-195" - Thread t@989
java.lang.Thread.State: RUNNABLE
at
org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
at
org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
at
org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
at
org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
at
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:156)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:146)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:125)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:99)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:263)
at
[jira] [Updated] (CASSANDRA-13432) MemtableReclaimMemory can get stuck because of lack of timeout in getTopLevelColumns()
[
https://issues.apache.org/jira/browse/CASSANDRA-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Corentin Chary updated CASSANDRA-13432:
---
Description:
This might affect 3.x too, I'm not sure.
{code}
$ nodetool tpstats
Pool NameActive Pending Completed Blocked All
time blocked
MutationStage 0 0 32135875 0
0
ReadStage 114 0 29492940 0
0
RequestResponseStage 0 0 86090931 0
0
ReadRepairStage 0 0 166645 0
0
CounterMutationStage 0 0 0 0
0
MiscStage 0 0 0 0
0
HintedHandoff 0 0 47 0
0
GossipStage 0 0 188769 0
0
CacheCleanupExecutor 0 0 0 0
0
InternalResponseStage 0 0 0 0
0
CommitLogArchiver 0 0 0 0
0
CompactionExecutor0 0 86835 0
0
ValidationExecutor0 0 0 0
0
MigrationStage0 0 0 0
0
AntiEntropyStage 0 0 0 0
0
PendingRangeCalculator0 0 92 0
0
Sampler 0 0 0 0
0
MemtableFlushWriter 0 0563 0
0
MemtablePostFlush 0 0 1500 0
0
MemtableReclaimMemory 129534 0
0
Native-Transport-Requests41 0 54819182 0
1896
{code}
{code}
"MemtableReclaimMemory:195" - Thread t@6268
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:283)
at
org.apache.cassandra.utils.concurrent.OpOrder$Barrier.await(OpOrder.java:417)
at
org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1151)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- locked <6e7b1160> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"SharedPool-Worker-195" - Thread t@989
java.lang.Thread.State: RUNNABLE
at
org.apache.cassandra.db.RangeTombstoneList.addInternal(RangeTombstoneList.java:690)
at
org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:650)
at
org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:171)
at
org.apache.cassandra.db.RangeTombstoneList.add(RangeTombstoneList.java:143)
at org.apache.cassandra.db.DeletionInfo.add(DeletionInfo.java:240)
at
org.apache.cassandra.db.ArrayBackedSortedColumns.delete(ArrayBackedSortedColumns.java:483)
at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:153)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:184)
at
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:156)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:146)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:125)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:99)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:263)
at
