[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196296#comment-14196296
 ] 

Ariel Weisberg edited comment on CASSANDRA-7438 at 11/4/14 4:13 PM:
--------------------------------------------------------------------

bq. No we don't. We have locks per Segment, this is very similar to lock 
stripping/Java's concurrent hash map.
Thanks for clearing that up

bq. Not really we lock globally when we reach 100% of the space and we freeup 
to 80% of the space and we spread the overhead to other threads based on who 
ever has the item partition lock. It won't be hard to make this part of the 
queue thread and will try it for the next release of lruc.

OK, that make sense. 20% of the cache could be many milliseconds of work if you 
are using many gigabytes of cache. That's not a great thing to foist on a 
random victim thread. If you handed that to the queue thread, well I think you 
run into another issue which is that the ring buffer doesn't appear to check 
for queue full? The queue thread could go out to lunch for a while. Not a big 
deal, but finer grained scheduling will probably be necessary.

bq. If you look at the code closer to memcached. Actually I started of 
stripping memcached code so we can run it in process instead of running as a 
separate process and removing the global locks in queue reallocation etc and 
eventually diverged too much from it. The other reason it doesn't use slab 
allocators is because we wanted the memory allocators to do the right thing we 
already have tested Cassandra with Jemalloc.

Ah very cool.

jemalloc is not a moving allocator where as it looks like memcached slabs 
implement rebalancing to accommodate changes in size distribution. That would 
actually be one of the really nice things to keep IMO. On large memory systems 
with a cache that scales and performs you would end up dedicating as much RAM 
as possible to the row cache/key cache and not the page cache since the page 
cache is not as granular (correct me if the story for C* is different). If you 
dedicate 80% of RAM to the cache that doesn't leave a lot of space left for 
fragmentation. By using a heap allocator you also lose the ability to implement 
hard predictable limits on memory used by the cache since you didn't map it 
yourself. I could be totally off base and jemalloc might be good enough.

bq. There is some comments above which has the reasoning for it (please see the 
above comments). PS: I believe there was some tickets on Current RowCache 
complaining about the overhead.
I don't have a performance beef with JNI, especially the way you have done 
which I think is pretty efficient. I think the overhead of JNI (one or two 
slightly more expensive function calls) would be eclipsed by things like the 
cache misses, coherence, and pipeline stalls that are part of accessing and 
maintaining a concurrent cache (Java or C++). It's all just intuition without 
comparative microbenchmarks of the two caches. Java might look a little faster 
just due to allocator performance, but we know you pay for that in other ways.

I think what you have made scratches the itch for a large cache quite well, and 
beats the status quo. I don't agree that Unsafe couldn't do the exact same 
thing with no on heap references.

The hash table, ring buffer, and individual item entries are all being malloced 
and you can do that from Java using Unsafe. You don't need to implement a ring 
buffer because you can use Disruptor. I also wonder if splitting the cache into 
several instances each with a coarse lock per instance wouldn't result in 
simpler, and I know performance is not an issue, fast enough code. I don't want 
to advocate doing something different for performance, but rather that there is 
the possibility of a relatively simple implementation via Unsafe.

You could coalesce all the contended fields for each instance (stats, lock 
field, LRU head) into a single cache line, and then rely on a single barrier 
when releasing a coarse grained lock. The fine grained locking and CASing 
results in several pipeline stalls because the memory barriers that are 
implicit in each one require the store buffers to drain. There may even be a 
suitable off heap map implementation out there already.


was (Author: aweisberg):
.bq No we don't. We have locks per Segment, this is very similar to lock 
stripping/Java's concurrent hash map.
Thanks for clearing that up

.bq Not really we lock globally when we reach 100% of the space and we freeup 
to 80% of the space and we spread the overhead to other threads based on who 
ever has the item partition lock. It won't be hard to make this part of the 
queue thread and will try it for the next release of lruc.

OK, that make sense. 20% of the cache could be many milliseconds of work if you 
are using many gigabytes of cache. That's not a great thing to foist on a 
random victim thread. If you handed that to the queue thread, well I think you 
run into another issue which is that the ring buffer doesn't appear to check 
for queue full? The queue thread could go out to lunch for a while. Not a big 
deal, but finer grained scheduling will probably be necessary.

.bq If you look at the code closer to memcached. Actually I started of 
stripping memcached code so we can run it in process instead of running as a 
separate process and removing the global locks in queue reallocation etc and 
eventually diverged too much from it. The other reason it doesn't use slab 
allocators is because we wanted the memory allocators to do the right thing we 
already have tested Cassandra with Jemalloc.

Ah very cool.

jemalloc is not a moving allocator where as it looks like memcached slabs 
implement rebalancing to accommodate changes in size distribution. That would 
actually be one of the really nice things to keep IMO. On large memory systems 
with a cache that scales and performs you would end up dedicating as much RAM 
as possible to the row cache/key cache and not the page cache since the page 
cache is not as granular (correct me if the story for C* is different). If you 
dedicate 80% of RAM to the cache that doesn't leave a lot of space left for 
fragmentation. By using a heap allocator you also lose the ability to implement 
hard predictable limits on memory used by the cache since you didn't map it 
yourself. I could be totally off base and jemalloc might be good enough.

.bq There is some comments above which has the reasoning for it (please see the 
above comments). PS: I believe there was some tickets on Current RowCache 
complaining about the overhead.
I don't have a performance beef with JNI, especially the way you have done 
which I think is pretty efficient. I think the overhead of JNI (one or two 
slightly more expensive function calls) would be eclipsed by things like the 
cache misses, coherence, and pipeline stalls that are part of accessing and 
maintaining a concurrent cache (Java or C++). It's all just intuition without 
comparative microbenchmarks of the two caches. Java might look a little faster 
just due to allocator performance, but we know you pay for that in other ways.

I think what you have made scratches the itch for a large cache quite well, and 
beats the status quo. I don't agree that Unsafe couldn't do the exact same 
thing with no on heap references.

The hash table, ring buffer, and individual item entries are all being malloced 
and you can do that from Java using Unsafe. You don't need to implement a ring 
buffer because you can use Disruptor. I also wonder if splitting the cache into 
several instances each with a coarse lock per instance wouldn't result in 
simpler, and I know performance is not an issue, fast enough code. I don't want 
to advocate doing something different for performance, but rather that there is 
the possibility of a relatively simple implementation via Unsafe.

You could coalesce all the contended fields for each instance (stats, lock 
field, LRU head) into a single cache line, and then rely on a single barrier 
when releasing a coarse grained lock. The fine grained locking and CASing 
results in several pipeline stalls because the memory barriers that are 
implicit in each one require the store buffers to drain. There may even be a 
suitable off heap map implementation out there already.

> Serializing Row cache alternative (Fully off heap)
> --------------------------------------------------
>
>                 Key: CASSANDRA-7438
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>         Environment: Linux
>            Reporter: Vijay
>            Assignee: Vijay
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: 0001-CASSANDRA-7438.patch
>
>
> Currently SerializingCache is partially off heap, keys are still stored in 
> JVM heap as BB, 
> * There is a higher GC costs for a reasonably big cache.
> * Some users have used the row cache efficiently in production for better 
> results, but this requires careful tunning.
> * Overhead in Memory for the cache entries are relatively high.
> So the proposal for this ticket is to move the LRU cache logic completely off 
> heap and use JNI to interact with cache. We might want to ensure that the new 
> implementation match the existing API's (ICache), and the implementation 
> needs to have safe memory access, low overhead in memory and less memcpy's 
> (As much as possible).
> We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to