Thank you. Some more follow-up questions 1) great, will do some tests
2) if auto commit is used how do we prevent a commit happening when an error happens in processing. Basically our scenario is that we build up aggregation contexts for specific keys (these are a bit special so most probably can't use KTables) and we would then on each punctuate call want to save these contexts to our external systems. Once saved we would commit our offset. However if the progress is committed before punctuate and we have an error saving we could end up with the offset being ahead of our saved progress. 3) great 4) great 5) What happens if a process/punctuate call throws a RuntimeException? Regards Toby > On 27 May 2016, at 1:32 AM, Matthias J. Sax <matth...@confluent.io> wrote: > > Hi Toby, > > 1) I am not an expert for RocksDB, but I don't see a problem with larger > objects. > > 2) I assume, by "guaranteed" you mean that the commit is performed when > the call return. In this case, no. It only sets a flag to commit at the > next earliest point in time possible. Ie, you can trigger commits in > between the regular commit interval that is configured via > "commit.interval.ms" > > 3) Yes. > > 4) Yes. > > -Matthias > > > On 05/26/2016 02:36 PM, Tobias Adamson wrote: >> Hi >> We have a scenario where we could benefit from the new API’s instead of our >> in house ones. >> However we have a couple of questions >> >> 1. Is it feasible to save 2-3MB size values in the RocksDBStorage? >> 2. When is the offset committed as processed when using a custom Processor, >> is it when you call commit on the context and is the commit guaranteed? >> 3. Is it ok to put values in to the KVStore in the punctuate method? >> 4. Is the punctuate method run by the same thread as the process method? >> >> >> Regards >> Toby >> >
