[jira] [Commented] (CASSANDRA-13553) Map C* Wide column to RocksDB key value data model
[ https://issues.apache.org/jira/browse/CASSANDRA-13553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188632#comment-16188632 ] DOAN DuyHai commented on CASSANDRA-13553: - I like the idea of having special encoding to preserve the orders of clustering columns. Neat! About point 2 of storing all normal columns of C* into RocksDB value, how would we deal with CQL mutation on a single column of the row ? Would it require necessarily a read-before-write or do we have smarter alternatives ? If the read-before-write is not avoidable, maybe clearly document the performance trade-off when using RockDBs for write so that users don't get surprised. > Map C* Wide column to RocksDB key value data model > -- > > Key: CASSANDRA-13553 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13553 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu > > The goal for this ticket is to find a way to map Cassandra's wide column data > model to RocksDB's key value data model. > To support most common C* queries on top of RocksDB, we plan to use this > strategy, for each row in Cassandra: > 1. Encode Cassandra partition key + clustering keys into RocksDB key. > 2. Encode rest of Cassandra columns into RocksDB value. > With this approach, there are two major problems we need to solve: > 1. After we encode C* keys into RocksDB key, we need to preserve the same > sorting order in RocksDB byte comparator, as in original data type. > 2. Support timestamp, ttl, and tombestone on the values. > To solve problem 1, we need to carefully design the encoding algorithm for > each data type. Fortunately, there are some existing libraries we can play > with, such as orderly (https://github.com/ndimiduk/orderly), which is used by > HBase. Or flatbuffer (https://github.com/google/flatbuffers) > To solve problem 2, our plan is to encode C* timestamp, ttl, and tombestone > together with the values, and then use RocksDB's merge operator/compaction > filter to merge different version of data, and handle ttl/tombestones. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13553) Map C* Wide column to RocksDB key value data model
[ https://issues.apache.org/jira/browse/CASSANDRA-13553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188516#comment-16188516 ] Aleksey Yeschenko commented on CASSANDRA-13553: --- Don't forget row and range tombstones, too. > Map C* Wide column to RocksDB key value data model > -- > > Key: CASSANDRA-13553 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13553 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu > > The goal for this ticket is to find a way to map Cassandra's wide column data > model to RocksDB's key value data model. > To support most common C* queries on top of RocksDB, we plan to use this > strategy, for each row in Cassandra: > 1. Encode Cassandra partition key + clustering keys into RocksDB key. > 2. Encode rest of Cassandra columns into RocksDB value. > With this approach, there are two major problems we need to solve: > 1. After we encode C* keys into RocksDB key, we need to preserve the same > sorting order in RocksDB byte comparator, as in original data type. > 2. Support timestamp, ttl, and tombestone on the values. > To solve problem 1, we need to carefully design the encoding algorithm for > each data type. Fortunately, there are some existing libraries we can play > with, such as orderly (https://github.com/ndimiduk/orderly), which is used by > HBase. Or flatbuffer (https://github.com/google/flatbuffers) > To solve problem 2, our plan is to encode C* timestamp, ttl, and tombestone > together with the values, and then use RocksDB's merge operator/compaction > filter to merge different version of data, and handle ttl/tombestones. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13553) Map C* Wide column to RocksDB key value data model
[ https://issues.apache.org/jira/browse/CASSANDRA-13553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023900#comment-16023900 ] Dikang Gu commented on CASSANDRA-13553: --- This will include changes both in Cassandra and RocksDB, we already submitted a PR for the rocksdb merge operator changes, https://github.com/facebook/rocksdb/pull/2289/files > Map C* Wide column to RocksDB key value data model > -- > > Key: CASSANDRA-13553 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13553 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Dikang Gu >Assignee: Dikang Gu > > The goal for this ticket is to find a way to map Cassandra's wide column data > model to RocksDB's key value data model. > To support most common C* queries on top of RocksDB, we plan to use this > strategy, for each row in Cassandra: > 1. Encode Cassandra partition key + clustering keys into RocksDB key. > 2. Encode rest of Cassandra columns into RocksDB value. > With this approach, there are two major problems we need to solve: > 1. After we encode C* keys into RocksDB key, we need to preserve the same > sorting order in RocksDB byte comparator, as in original data type. > 2. Support timestamp, ttl, and tombestone on the values. > To solve problem 1, we need to carefully design the encoding algorithm for > each data type. Fortunately, there are some existing libraries we can play > with, such as orderly (https://github.com/ndimiduk/orderly), which is used by > HBase. Or flatbuffer (https://github.com/google/flatbuffers) > To solve problem 2, our plan is to encode C* timestamp, ttl, and tombestone > together with the values, and then use RocksDB's merge operator/compaction > filter to merge different version of data, and handle ttl/tombestones. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
