[
https://issues.apache.org/jira/browse/SAMZA-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359728#comment-14359728
]
Chris Riccomini commented on SAMZA-543:
---------------------------------------
bq. should disableDataSync be set to true ? I couldn't figure out the default
value used in the JNI bindings.
We should force this to true. Looks like I was mistaken. disableDataSync
[defaults to
false|https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/DBOptionsInterface.java#L228]
according to the Javadocs.
bq. Do we want to run any performance tests before we commit this change?
I think so, just to prove that it's still faster with the WAL disabled. Maybe
you can modify TestKeyValuePerformance to make it useful? Right now it just
tests a very specific use case (all with deletes). It'd be nice to test bulk
loading some data both with/without WAL/data sync, and see how it performs.
bq. My suggestion is to do this one change at a time and measure the
performance improvement.
Agreed.
> Disable WAL in RocksDB KV store
> -------------------------------
>
> Key: SAMZA-543
> URL: https://issues.apache.org/jira/browse/SAMZA-543
> Project: Samza
> Issue Type: Bug
> Components: kv
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
> Assignee: Navina Ramesh
> Fix For: 0.9.0
>
> Attachments: SAMZA-543-0.patch
>
>
> RocksDB uses a write-ahead log by default. This is unnecessary in Samza,
> since we have full durability from a state store's changelog topic. We should
> [disable the
> WAL|https://github.com/facebook/rocksdb/wiki/Basic-Operations#asynchronous-writes]
> in the RocksDB KV store.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
