[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16261: --- Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.0.24 3.11.10 > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.24, 3.11.10, 4.0-rc1, 4.0 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16261: --- Fix Version/s: (was: 4.0-beta) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta5, 4.0, 3.0.x, 3.11.x > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Test and Documentation Plan: Patch committed: * trunk - [1b4e1cc9303414f91535dab10f6342e1c1c6b8ac|https://github.com/apache/cassandra/commit/1b4e1cc9303414f91535dab10f6342e1c1c6b8ac] - 3.11 - [11cb8104facc4d3455efb73d2701fd5ba510bef1|https://github.com/apache/cassandra/commit/11cb8104facc4d3455efb73d2701fd5ba510bef1] - 3.0 - [0a1e900a0a042f78d7d5d6625bc98b84eb463e69|https://github.com/apache/cassandra/commit/0a1e900a0a042f78d7d5d6625bc98b84eb463e69] was:https://issues.apache.org/jira/browse/CASSANDRA-16261?focusedCommentId=17262315=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17262315 > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Source Control Link: https://github.com/apache/cassandra/commit/0a1e900a0a042f78d7d5d6625bc98b84eb463e69 > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Since Version: (was: 2.2.0) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Since Version: 2.2.0 (was: 3.0.0) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Source Control Link: (was: Patch committed: trunk - 1b4e1cc9303414f91535dab10f6342e1c1c6b8ac 3.11 - 11cb8104facc4d3455efb73d2701fd5ba510bef1 3.0 - 0a1e900a0a042f78d7d5d6625bc98b84eb463e69) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Since Version: 3.0.0 Source Control Link: Patch committed: trunk - 1b4e1cc9303414f91535dab10f6342e1c1c6b8ac 3.11 - 11cb8104facc4d3455efb73d2701fd5ba510bef1 3.0 - 0a1e900a0a042f78d7d5d6625bc98b84eb463e69 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16261: -- Status: Ready to Commit (was: Review In Progress) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16261: Reviewers: Andres de la Peña, Caleb Rackliffe, Caleb Rackliffe (was: Andres de la Peña, Caleb Rackliffe) Andres de la Peña, Caleb Rackliffe, Caleb Rackliffe (was: Andres de la Peña, Caleb Rackliffe) Status: Review In Progress (was: Patch Available) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Test and Documentation Plan: https://issues.apache.org/jira/browse/CASSANDRA-16261?focusedCommentId=17262315=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17262315 (was: https://issues.apache.org/jira/browse/CASSANDRA-16261?focusedCommentId=17231874=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17231874) Status: Patch Available (was: In Progress) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16261: --- Fix Version/s: (was: 4.0-beta4) 4.0-beta > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Status: Changes Suggested (was: Review In Progress) Drop the parameter as root cause was fixed, confirmed. Metric can be added to improve system monitoring. > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Status: In Progress (was: Changes Suggested) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Fix Version/s: (was: 2.2.x) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Description: The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty. This patch adds a mechanism to track the number of pending flushing tasks in the memtable cleaner. Above the maximum number (2x the flushing threads by default), only memtables using at least MCT memory will be flushed, where MCT stands for Memory Cleanup Threshold. This patch also fixes a possible problem tracking the memory marked as "reclaiming" in the memtable allocators and pool. Writes that complete only after a memtable has been scheduled for flushing, did not report their memory as reclaiming. Normally this should be a small value of no consequence, but if the flushing tasks are blocked for a long period, and there is a sufficient number of writes, or these writes use a sufficiently large quantity of memory, this would cause the memtable cleaning algorithm to schedule repeated flushing tasks because the used memory is always > reclaiming memory + MCT. was:The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty. > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. > This patch adds a mechanism to track the number of pending flushing > tasks in the memtable cleaner. Above the maximum number (2x the flushing > threads by default), only memtables using at least MCT memory will be > flushed, where MCT stands for Memory Cleanup Threshold. > This patch also fixes a possible problem tracking the memory marked as > "reclaiming" in the memtable allocators and pool. Writes that complete > only after a memtable has been scheduled for flushing, did not report > their memory as reclaiming. Normally this should be a small value of no > consequence, but if the flushing tasks are blocked for a long period, > and there is a sufficient number of writes, or these writes use > a sufficiently large quantity of memory, this would cause the memtable > cleaning algorithm to schedule repeated flushing tasks because the used > memory is always > reclaiming memory + MCT. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Fix Version/s: 3.0.x 2.2.x > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16261: Reviewers: Andres de la Peña, Caleb Rackliffe (was: Andres de la Peña) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16261: -- Reviewers: Andres de la Peña, Andres de la Peña (was: Andres de la Peña) Andres de la Peña, Andres de la Peña (was: Andres de la Peña) Status: Review In Progress (was: Patch Available) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16261: -- Reviewers: Andres de la Peña > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Test and Documentation Plan: https://issues.apache.org/jira/browse/CASSANDRA-16261?focusedCommentId=17231874=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17231874 Status: Patch Available (was: In Progress) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Fix Version/s: (was: 3.0.24) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Fix Version/s: 3.0.24 > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4, 3.0.24 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Description: The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty. > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > > The cleaner thread is not prevented from queueing an unbounded number of > flushing tasks for memtables that are almost empty. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16261: Bug Category: Parent values: Code(13163) Complexity: Normal Component/s: Legacy/Local Write-Read Paths Discovered By: User Report Fix Version/s: 4.0-beta4 3.11.x Severity: Normal Assignee: Ekaterina Dimitrova Status: Open (was: Triage Needed) > Prevent unbounded number of flushing tasks > -- > > Key: CASSANDRA-16261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16261 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
