[
https://issues.apache.org/jira/browse/CASSANDRA-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878723#action_12878723
]
Ben Manes commented on CASSANDRA-975:
-------------------------------------
In CHM, the concurrency level is the number of concurrent writer threads
(sharded by segments) so 64 is quite high. In CLHM, there is a read queue per
segment to buffer LRU reorder operations. The buffer is drained when it exceeds
a threshold (64 entries) or a write occurs. This probably accounts for the
additional memory overhead, as there is a max of 64x64=4096 entries that may be
buffered. The reason it is per segment is that reads are by far the common
case, so if there are a large number of writers there should be far more
readers. So the increased number of buffers avoids contending on the queue but
adds memory overhead.
A flaw in v1.0 is that only one segment's buffer is drained at a time. Ideally
when the lock is acquired (to drain the buffer, apply writes, and optionally
evict) it should drain all of the read buffers. This would reduce memory and
improve the hit rate, which would probably resolve the regression issues. This
was fixed in /trunk, but it can be back-ported.
I also think that I can improve the write performance by removing the lock.
This is needed to maintain a strict write ordering on the write queue to match
the segment's, but I believe that with a little magic I can remove that
constraint. That would make a minor boost.
I'd be happy to work on a v1.1-LRU release that focuses on performance tuning.
> explore upgrading CLHM
> ----------------------
>
> Key: CASSANDRA-975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-975
> Project: Cassandra
> Issue Type: Task
> Reporter: Jonathan Ellis
> Assignee: Matthew F. Dennis
> Priority: Minor
> Fix For: 0.8
>
> Attachments: 0001-trunk-975.patch, clhm_test_results.txt,
> insertarator.py, readarator.py
>
>
> The new version should be substantially better "on large caches where many
> entries were readon large caches where many entries were read," which is
> exactly what you see in our row and key caches.
> http://code.google.com/p/concurrentlinkedhashmap/issues/detail?id=9
> Hopefully we can get Digg to help test, since they could reliably break CLHM
> when it was buggy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
