[
https://issues.apache.org/jira/browse/CASSANDRA-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048624#comment-13048624
]
Pavel Yaskevich commented on CASSANDRA-2731:
--------------------------------------------
RadixTree is the best data structure for that - tree of the low depth allows to
store and fast access to the huge amount of data (for 4KB page, 64 slots in
each of the nodes on the depth of 6 allows to store/index up to 15TB of data),
it will be implemented in Java so no problem with portability.
http://lwn.net/Articles/175432/ overview of the structure
> Impelement in-house file caching.
> ---------------------------------
>
> Key: CASSANDRA-2731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2731
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 1.0
> Reporter: Pavel Yaskevich
> Assignee: Pavel Yaskevich
>
> Implement FileCache, CachedRandomAccessFile (to replace
> BufferedRandomAccessFile) and RadixTree (to play role of the backend cache
> storage) classes.
> FileCache class with be responsible for storing/retrieving data from Radix
> Tree and also flushing of the dirty pages to the disk, page management such
> as adding new pages, utilizing old/unused pages.
> CRAF Linux only features (via JNI):
> 1). O_DIRECT for both read/write operations.
> 2). AIO's lio_listio write operation batching.
> Provide possibility to migrate hot data directly from Memtable to CRAF cache
> to keep live-reads data always hot in memory. To minimise compaction effects
> CRAF should provide a way to by-pass a caching data if it does not already
> exists.
> Provide a way to make pointers in the cache which will be useful to minimize
> impact on performance when a single column is distributed among multiple
> SSTable files (except counter columns).
> Use jemalloc (http://www.canonware.com/jemalloc/) for cache memory management.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
