[ https://issues.apache.org/jira/browse/CASSANDRA-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Thompson updated CASSANDRA-8409: --------------------------------------- Reproduced In: 2.1.0 Fix Version/s: 2.1.3 > Node generating a huge number of tiny sstable_activity flushes > -------------------------------------------------------------- > > Key: CASSANDRA-8409 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8409 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.1.0, Oracle JDK 1.8.0_25, Ubuntu 12.04 > Reporter: Fred Wulff > Fix For: 2.1.3 > > Attachments: system-sstable_activity-ka-67802-Data.db > > > On one of my nodes, I’m seeing hundreds per second of “INFO 21:28:05 > Enqueuing flush of sstable_activity: 0 (0%) on-heap, 33 (0%) off-heap”. > tpstats shows a steadily climbing # of pending > MemtableFlushWriter/MemtablePostFlush until the node OOMs. When the flushes > actually happen the sstable written is invariably 121 bytes. I’m writing > pretty aggressively to one of my user tables (sev.mdb_group_pit), but that > table's flushing behavior seems reasonable. > tpstats: > {quote} > frew@hostname:~/s_dist/apache-cassandra-2.1.0$ bin/nodetool -h hostname > tpstats > Pool Name Active Pending Completed Blocked All > time blocked > MutationStage 128 4429 36810 0 > 0 > ReadStage 0 0 1205 0 > 0 > RequestResponseStage 0 0 24910 0 > 0 > ReadRepairStage 0 0 26 0 > 0 > CounterMutationStage 0 0 0 0 > 0 > MiscStage 0 0 0 0 > 0 > HintedHandoff 2 2 9 0 > 0 > GossipStage 0 0 5157 0 > 0 > CacheCleanupExecutor 0 0 0 0 > 0 > InternalResponseStage 0 0 0 0 > 0 > CommitLogArchiver 0 0 0 0 > 0 > CompactionExecutor 4 28 429 0 > 0 > ValidationExecutor 0 0 0 0 > 0 > MigrationStage 0 0 0 0 > 0 > AntiEntropyStage 0 0 0 0 > 0 > PendingRangeCalculator 0 0 11 0 > 0 > MemtableFlushWriter 8 38644 8987 0 > 0 > MemtablePostFlush 1 38940 8735 0 > 0 > MemtableReclaimMemory 0 0 8987 0 > 0 > Message type Dropped > READ 0 > RANGE_SLICE 0 > _TRACE 0 > MUTATION 10457 > COUNTER_MUTATION 0 > BINARY 0 > REQUEST_RESPONSE 0 > PAGED_RANGE 0 > READ_REPAIR 208 > {quote} > I've attached one of the produced sstables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)