[
https://issues.apache.org/jira/browse/CASSANDRA-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989549#comment-12989549
]
Peter Schuller commented on CASSANDRA-1657:
-------------------------------------------
Depending on use-case maybe, but I don't think always. Primarily for three
reasons:
(1) The row cache is not at all as efficient in terms of memory use. Assuming
the premise that you really want it to be guaranteed in memory is true (rather
than a 10% active set), the on-disk format is considerably more compact (more
so for very small rows with very small columns, than for medium-sized cf:s with
less keys). A nice bonus is avoiding GC pressure, and edge cases like the odd
large row and difficulty sizing the row cache.
(2) The row-cache has a sever trade-off in terms of start-up time due to it
being seek-bound (although for CF:s that are supposed to be in-memory anyway it
may be mitigated by pre-heating it by streaming through it; but that feels more
like a hack). A kernel-driven sequential pre-population is vastly preferred ;)
(3) Hopefully with future improvements in compaction (e.g. limiting maximum
sstable size so that compaction can happen to reasonably sized chunks, thus not
doubling memory use) an in-memory CF would stay in-memory completely during
compaction (only incurring deferred writes). In other words, it would
contribute significantly less to (disk i/o) compaction overhead than if the
active set was in row cache.
I do assume that the use-case is such that optimizing for avoiding
serialization/deserialization is not the issue here, and that the importance of
keeping the data off disk is more important than squeezing the last bit of CPU
efficiency out of it.
Since Cassandra is going the route of native code and embracing things like
posix_fadvise/mincore/mlock etc anyway, it seemed like a reasonable thing to
support. However of course if other features end up catering sufficiently to
the same goal, the suggestion may become obsolete.
Also, I still like the stuff in CASSANDRA-1625 (specifically
https://issues.apache.org/jira/browse/CASSANDRA-1625?focusedCommentId=12971877&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12971877)
which overlaps with this.
> support in-memory column families
> ---------------------------------
>
> Key: CASSANDRA-1657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1657
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Peter Schuller
> Priority: Minor
>
> Some workloads are such that you absolutely depend on column families being
> in-memory for performance, yet you most definitely want all the things that
> Cassandra offers in terms of replication, consistency, durability etc.
> In order to semi-deterministically ensure acceptable performance for such
> data, Cassandra could support in-memory column families. Such an in-memory
> column family would imply that mlock() be used on sstables for this column
> family. On start-up and on compaction completion, they could be mmap():ed
> with MAP_POPULATE (Linux specific) or else just mmap():ed + mlock():ed in
> such a way as to otherwise guarantee it is in-memory (such as userland
> traversal of the entire file).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
