[jira] [Commented] (CASSANDRA-13553) Map C* table schema to RocksDB key value data model
[ https://issues.apache.org/jira/browse/CASSANDRA-13553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189682#comment-16189682 ] DOAN DuyHai commented on CASSANDRA-13553: - Thanks [~dikanggu], can't wait to play with this new storage engine once we have a beta version > Map C* table schema to RocksDB key value data model > --- > > Key: CASSANDRA-13553 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13553 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu > > The goal for this ticket is to find a way to map Cassandra's table data model > to RocksDB's key value data model. > To support most common C* queries on top of RocksDB, we plan to use this > strategy, for each row in Cassandra: > 1. Encode Cassandra partition key + clustering keys into RocksDB key. > 2. Encode rest of Cassandra columns into RocksDB value. > With this approach, there are two major problems we need to solve: > 1. After we encode C* keys into RocksDB key, we need to preserve the same > sorting order in RocksDB byte comparator, as in original data type. > 2. Support timestamp, ttl, and tombestone on the values. > To solve problem 1, we need to carefully design the encoding algorithm for > each data type. Fortunately, there are some existing libraries we can play > with, such as orderly (https://github.com/ndimiduk/orderly), which is used by > HBase. Or flatbuffer (https://github.com/google/flatbuffers) > To solve problem 2, our plan is to encode C* timestamp, ttl, and tombestone > together with the values, and then use RocksDB's merge operator/compaction > filter to merge different version of data, and handle ttl/tombestones. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13553) Map C* table schema to RocksDB key value data model
[ https://issues.apache.org/jira/browse/CASSANDRA-13553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188906#comment-16188906 ] Dikang Gu commented on CASSANDRA-13553: --- [~doanduyhai], we do not need to do read-before-write, any mutation will be a new RocksDB row, and we will merge the data in read path or during compaction, through RocksDB's merge operator. > Map C* table schema to RocksDB key value data model > --- > > Key: CASSANDRA-13553 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13553 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu > > The goal for this ticket is to find a way to map Cassandra's table data model > to RocksDB's key value data model. > To support most common C* queries on top of RocksDB, we plan to use this > strategy, for each row in Cassandra: > 1. Encode Cassandra partition key + clustering keys into RocksDB key. > 2. Encode rest of Cassandra columns into RocksDB value. > With this approach, there are two major problems we need to solve: > 1. After we encode C* keys into RocksDB key, we need to preserve the same > sorting order in RocksDB byte comparator, as in original data type. > 2. Support timestamp, ttl, and tombestone on the values. > To solve problem 1, we need to carefully design the encoding algorithm for > each data type. Fortunately, there are some existing libraries we can play > with, such as orderly (https://github.com/ndimiduk/orderly), which is used by > HBase. Or flatbuffer (https://github.com/google/flatbuffers) > To solve problem 2, our plan is to encode C* timestamp, ttl, and tombestone > together with the values, and then use RocksDB's merge operator/compaction > filter to merge different version of data, and handle ttl/tombestones. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
