[
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140093#comment-14140093
]
Benedict edited comment on CASSANDRA-7282 at 9/19/14 7:01 AM:
--------------------------------------------------------------
We should test both. This change is going to apply not only now, but in the
future. If we reject it, it won't be revisited for a long time. Not only are VM
improvements like shenandoah and G1GC going to mean we support users with huge
heaps that can support gigantic memtables, this data structure is easily moved
completely offheap once the objects it refers to are. Lots of users run with
only single disks and large memory capacities also, and with a single disk an
mct of 0.5 is the norm. So even an mct of 0.99 is pretty underwhelming for
helping to bench these cases. An mct of 0.11 is completely inadequate.
However we should test both if that's what you run with, to see how it fairs in
different climates.
(FTR: my 'realistic' runs were with an mct of 0.4)
was (Author: benedict):
We should test both. This change is going to apply not only now, but in the
future. If we reject it, it won't be revisited for a long time. Not only are VM
improvements like shenandoah and G1GC going to mean we support users with huge
heaps that can support gigantic memtables, this data structure is easily moved
completely offheap once the objects it refers to are. Lots of users run with
only single disks and large memory capacities also, and with a single disk an
mct of 0.5 is the norm. So even an mct of 0.99 is pretty underwhelming for
helping to bench these cases. An mct of 0.11 is completely inadequate.
However we should test both if that's what you run with, to see how it fairs in
different climates.
> Faster Memtable map
> -------------------
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Benedict
> Assignee: Benedict
> Labels: performance
> Fix For: 3.0
>
> Attachments: profile.yaml, reads.svg, run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast
> majority of users use a hash partitioner, it occurs to me we could maintain a
> hybrid ordered list / hash map. The list would impose the normal order on the
> collection, but a hash index would live alongside as part of the same data
> structure, simply mapping into the list and permitting O(1) lookups and
> inserts.
> I've chosen to implement this initial version as a linked-list node per item,
> but we can optimise this in future by storing fatter nodes that permit a
> cache-line's worth of hashes to be checked at once, further reducing the
> constant factor costs for lookups.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
