[
https://issues.apache.org/jira/browse/CASSANDRA-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019381#comment-17019381
]
Benedict Elliott Smith edited comment on CASSANDRA-15511 at 1/22/20 10:50 PM:
------------------------------------------------------------------------------
I have uploaded the results of several days of performance comparisons, and for
posterity the scripts I used to generate and parse them into a spreadsheet, on
the effect of this change on inserting into a partition.
The results are pretty significant; some highlights:
Collection types are universally and dramatically improved, both contended and
uncontended:
* >99% memory consumption reduction
* ~25x throughput
Non-collection types, uncontended:
* ~25% improvement in throughput
* 50% reduction in memory allocated
Non-collections types, contended:
* vs 3.0 with locking enabled _instantly_ (i.e. upgrade to locking after first
competition)
** With 4 competing threads, across the board lower memory utilisation;
near-equivalent with 16-threads
** With 4 or 16 competing threads, ~200% throughput improvement
* vs 3.0 with default locking
** 100-200% throughput improvement
** 30-60% reduction in memory consumption
Notably, this may justify removing the partition lock entirely. This patch,
but with the lock retained, does see even fewer allocations, but the typical
effect is relatively small, and there is a similarly slight drop in throughput.
I recommend taking a look at the attached (zipped) Excel spreadsheet, which has
some nice visual formatting to see the full breakdown of results. I've also
attached as {{ods}} format for compatibility, but this loses the formatting and
inline graphical representation of improvement/degradation.
was (Author: benedict):
I have uploaded the results of several days of performance comparisons, and for
posterity the scripts I used to generate and parse them into a spreadsheet, on
the effect of this change on inserting into a partition.
The results are pretty significant; some highlights:
Collection types are universally and dramatically improved, both contended and
uncontended:
* >99% memory consumption reduction
* ~25x throughput
Non-collection types, uncontended:
* ~25% improvement in throughput
* 50% reduction in memory allocated
Non-collections types, contended:
* vs 3.0 with locking enabled _instantly_ (i.e. upgrade to locking after first
competition)
** With 4 competing threads, across the board lower memory utilisation;
near-equivalent with 16-threads
** With 4 or 16 competing threads, ~200% throughput improvement
* vs 3.0 with default locking
** 100-200% throughput improvement
** 30-60% reduction in memory consumption
Notably, this may justify removing the partition lock entirely. This patch,
but with the lock retained, does see even fewer allocations, but the typical
effect is relatively small, and there is a similarly slight drop in throughput.
I recommend taking a look at the attached (zipped) Excel spreadsheet, which has
some nice visual formatting to see the full breakdown of results. I've also
attached as {{ods}} format for compatibility, but this loses the formatting.
> Utilising BTree Improvements
> ----------------------------
>
> Key: CASSANDRA-15511
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15511
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Other
> Reporter: Benedict Elliott Smith
> Assignee: Benedict Elliott Smith
> Priority: Normal
> Fix For: 4.x
>
> Attachments: atomicbtreepartition.ods, atomicbtreepartition.xlsx.zip,
> perfsh.tar.gz
>
>
> This patch utilises CASSANDRA-15510 to improve throughput and reduce garbage
> produced by a number of common operations, by employing
> {{transformAndFilter}}, {{transform}} and {{FastBuilder}}
> * {{Row}}, {{Cell}} and {{ComplexColumnData}} cloning are implemented with
> {{BTree.transform}}, so no special builders are necessary;
> ** {{Rows.copy}} removed
> * {{Rows.merge}} implemented using {{BTree.update}} and a {{ColumnData}}
> reconciler
> ** Zero-allocations if result of merge is same as {{existing}}
> ** Fewer comparisons
> * {{ColumnData}} reconciler implemented in same manner
> ** {{Cells.reconcileComplex}} is retired
> ** {{ComplexColumnData}} reconciliation now
> *** Garbage-free if the merge has no effect
> *** Always fewer allocations
> *** Fewer comparisons
> * {{FastBuilder}} employed widely:
> ** {{ClusteringIndexNamesFilter}} deserialization
> ** {{Columns}} deserialization
> ** {{PartitionUpdate}} deserialization
> ** {{AbstractBTreePartition}} construction
> ** Misc others
> The upshot of this work when combined with the proposed patch for
> CASSANDRA-15367 has a dramatic impact on operations over collection types -
> under contention, as much as 100x improved throughput, and hundreds of
> megabytes of reduced allocations. For all operations, allocations under
> contention and no contention are significantly reduced and throughput
> improved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
