[ 
https://issues.apache.org/jira/browse/CASSANDRA-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506130#comment-15506130
 ] 

Benedict edited comment on CASSANDRA-12668 at 9/20/16 9:24 AM:
---------------------------------------------------------------

So, just to crystalise my thoughts on this.  Given this presents with 
contention, there are in all likelihood three possible causes:

* B-Tree is just too inherently worse here than SnapTreeMap was
* The extra work now done (besides b-tree) during memtable partition update 
increases the time taken, and hence number of concurrent operations
* One of the many other changes increases the number of concurrent operations 
(e.g. changes in threading or increased concurrent_writes), or the size of the 
partitions in a given memtable (e.g. fewer memtable flushes), is increasing the 
cost of contention

Ruling out the first one is probably most straightforwardly done by swapping 
back in SnapTreeMap in a patch and running your tests.  This wouldn't be 
trivial, but also should be far from terribly difficult, and would avoid 
striking around in the dark.

Otherwise, the suggestion I made in CASSANDRA-7546 stands as my preferred 
solution to this kind of problem: deferred updates under contention.  If a swap 
of head fails, just tag the update onto a queue, and have all readers merge the 
queue before responding, with all writers merging all deferred updates along 
with their own.  Readers may also update the b-tree with the result of their 
merge of there's no contention.



was (Author: benedict):
So, just to crystalise my thoughts on this.  Given this presents with 
contention, there are in all likelihood three possible causes:

* B-Tree is just too inherently worse here than SnapTreeMap was
* The extra work now done during memtable partition update increases the time 
taken, and hence number of concurrent operations
* One of the many other changes increases the number of concurrent operations 
(e.g. changes in threading or increased concurrent_writes), or the size of the 
partitions in a given memtable (e.g. fewer memtable flushes), is increasing the 
cost of contention

Ruling out the first one is probably most straightforwardly done by swapping 
back in SnapTreeMap in a patch and running your tests.  This wouldn't be 
trivial, but also should be far from terribly difficult, and would avoid 
striking around in the dark.

Otherwise, the suggestion I made in CASSANDRA-7546 stands as my preferred 
solution to this kind of problem: deferred updates under contention.  If a swap 
of head fails, just tag the update onto a queue, and have all readers merge the 
queue before responding, with all writers merging all deferred updates along 
with their own.  Readers may also update the b-tree with the result of their 
merge of there's no contention.


> Memtable Contention in 2.1
> --------------------------
>
>                 Key: CASSANDRA-12668
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12668
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: sankalp kohli
>
> We added a new Btree implementation in 2.1 which causes write performance to 
> go down in Cassandra if there is  lot of contention in the memtable for a CQL 
> partition. Upgrading a cluster from 2.0 to 2.1 with contention causes the 
> cluster to fall apart due to GC. We tried making the defaults added in 
> CASSANDRA-7546 configurable but that did not help. Is there anyway to fix 
> this issue?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to