[ https://issues.apache.org/jira/browse/KAFKA-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736113#comment-16736113 ]
Guozhang Wang commented on KAFKA-3184: -------------------------------------- [~Yohan123] I think you are right: currently `persistent()` is used to determine whether we should record checkpoint offsets, since for non-persistent stores there are no data flushed to persistent storage and therefore we can only restore from beginning every time upon resuming. With this JIRA in-memory stores would be "persisted" to storage as well, the only difference is that it is not written until that "snapshotting" point is reached (e.g. after N commits). And hence in this case persistent() flag is not that useful anyways and may consider deprecated. > Add Checkpoint for In-memory State Store > ---------------------------------------- > > Key: KAFKA-3184 > URL: https://issues.apache.org/jira/browse/KAFKA-3184 > Project: Kafka > Issue Type: Improvement > Components: streams > Reporter: Guozhang Wang > Priority: Major > Labels: user-experience > > Currently Kafka Streams does not make a checkpoint of the persistent state > store upon committing, which would be expensive since it is "stopping the > world" and write on disks: for example, RocksDB would require you to copy the > file directory to make a copy naively. > However, for in-memory stores checkpointing maybe doable in an asynchronous > manner hence it can be done quickly. And the benefit of having intermediate > checkpoint is to avoid restoring from scratch if standby tasks are not > present. -- This message was sent by Atlassian JIRA (v7.6.3#76005)