Hi Jack, Your understanding seems correct. The data is only considered durable if changelog is enabled. In particular the Host Affinity feature, which tries to maximize the reuse of on-disk state, only applies for change-logged stores. So with changelog disabled, you shouldn't expect state to be restored from disk.
The one additional thing that I'd point out is that the decision mostly depends on the size of your data. If your data is too large to fit in memory, then the RocksDB KVStore which flushes periodically to disk is still the best option because it usually requires significantly less memory. If however your data can fit entirely in memory, then the InMemory KVStore should provide the best performance, as it avoids marshalling to native memory and flushing to disk. -Jake On Tue, Aug 2, 2016 at 11:51 AM, Jack Huang <jackhu...@mz.com> wrote: > Hi all, > > Is there any reason to use RocksDB without associating it to changelog in > Kafka? My understanding is that even though Rocks persists data to disk, > when container fails the partition might be restarted on a different > machine, where there is no persisted data on disk. In that case, wouldn't > it make sense to use a in-memory store (e.g. java.util.Map) instead since > Rocks effectively does not provide persistence under Samza's architecture? > > Thanks, > Jack >
