prateekm commented on a change in pull request #1160: Transactional State
[1/5]: Added Store Checkpoint API
URL: https://github.com/apache/samza/pull/1160#discussion_r326761408
##########
File path:
samza-api/src/main/java/org/apache/samza/storage/kv/KeyValueStore.java
##########
@@ -141,4 +144,9 @@ default void deleteAll(List<K> keys) {
* Flushes this key-value store, if applicable.
*/
void flush();
+
+ /**
+ * Checkpoint the store snapshot.
+ */
+ Optional<Path> checkpoint(String id);
Review comment:
1. If None, changelog offsets will not be written to the store (e.g. for
persistent stores that don't support checkpoints) (PR #1164 ,
TransactionalStateTaskStorageManager). If exception, container will shut down.
I'll update the javadocs in #1164. I'll also add a test in #1164 to ensure that
any exception creating store checkpoint is propagated in
TransactionalStateTaskStorageManager. There's already a test that ensure any
exception thrown in TaskInstance#commit causes is propagated.
2. Transactional state is currently not supported for async commits. I'll
mark this method as unstable so that we can evolve it if necessary. The
behavior is: if checkpoints supported, return a path, else return None. On
errors, throw an exception. Not sure if separate status, result and exception
will help here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
