[
https://issues.apache.org/jira/browse/CASSANDRA-13474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dikang Gu updated CASSANDRA-13474:
----------------------------------
Description:
We did some experiment to switch Cassandra's storage engine to RocksDB.
In the experiment, I built a prototype to integrate Cassandra 3.0.12 and
RocksDB on single column (key-value) use case, shadowed one of our production
use case, and saw about 4-6X P99 read latency drop during peak time, compared
to 3.0.12. Also, the P99 latency became more predictable as well.
Here is detailed note with more metrics:
[https://docs.google.com/document/d/1Ztqcu8Jzh4USKoWBgDJQw82DBurQmsV-PmfiJYvu_Dc/edit?usp=sharing]
I think the biggest latency win comes from we get rid of most Java garbages
created by current read/write path and compactions, which reduces the JVM
overhead and makes the latency to be more predictable.
We are very excited about the potential performance gain. As the next step, I
propose to make the Cassandra storage engine to be pluggable (like Mysql and
MongoDB), and we are very interested in providing RocksDB as one storage option
with more predictable performance, together with community.
Design doc for pluggable storage engine:
https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc/edit
was:
We did some experiment to switch Cassandra's storage engine to RocksDB.
In the experiment, I built a prototype to integrate Cassandra 3.0.12 and
RocksDB on single column (key-value) use case, shadowed one of our production
use case, and saw about 4-6X P99 read latency drop during peak time, compared
to 3.0.12. Also, the P99 latency became more predictable as well.
Here is detailed note with more metrics:
https://docs.google.com/document/d/1Ztqcu8Jzh4USKoWBgDJQw82DBurQmsV-PmfiJYvu_Dc/edit?usp=sharing
I think the biggest latency win comes from we get rid of most Java garbages
created by current read/write path and compactions, which reduces the JVM
overhead and makes the latency to be more predictable.
We are very excited about the potential performance gain. As the next step, I
propose to make the Cassandra storage engine to be pluggable (like Mysql and
MongoDB), and we are very interested in providing RocksDB as one storage option
with more predictable performance, together with community.
> Cassandra pluggable storage engine
> ----------------------------------
>
> Key: CASSANDRA-13474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13474
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Dikang Gu
> Priority: Major
>
> We did some experiment to switch Cassandra's storage engine to RocksDB.
> In the experiment, I built a prototype to integrate Cassandra 3.0.12 and
> RocksDB on single column (key-value) use case, shadowed one of our production
> use case, and saw about 4-6X P99 read latency drop during peak time, compared
> to 3.0.12. Also, the P99 latency became more predictable as well.
> Here is detailed note with more metrics:
> [https://docs.google.com/document/d/1Ztqcu8Jzh4USKoWBgDJQw82DBurQmsV-PmfiJYvu_Dc/edit?usp=sharing]
> I think the biggest latency win comes from we get rid of most Java garbages
> created by current read/write path and compactions, which reduces the JVM
> overhead and makes the latency to be more predictable.
> We are very excited about the potential performance gain. As the next step, I
> propose to make the Cassandra storage engine to be pluggable (like Mysql and
> MongoDB), and we are very interested in providing RocksDB as one storage
> option with more predictable performance, together with community.
> Design doc for pluggable storage engine:
> https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc/edit
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
