Re: checkpoint example?

Stream name is the name of the topic that you are consuming from. (and the topic want to reset). On Thu, Mar 3, 2016 at 8:49 AM, Jeff Ramin wrote: > > Thank you! > > One more question - for the config provided below: > >

Re: checkpoint example?

Thank you! One more question - for the config provided below: systems.system-name.streams.stream-name.samza.reset.offset = true systems.system-name.streams.stream-name.samza.offset.default = oldest How do I determine what the "stream-name" is? I'm running the hello-samza example, which

Re: checkpoint example?

https://issues.apache.org/jira/browse/SAMZA-255 is the JIRA for rewinding a stream without restarting. On Thu, Mar 3, 2016 at 8:44 AM, Jagadish Venkatraman wrote: > You can use the checkpoint tool to publish the desired offset, and restart > your job. It will pick up

Re: checkpoint example?

Thanks Jacob. Regarding 2) below - is there a way to reprocess messages from an arbitrary position, instead of from the beginning? On 03/01/2016 06:32 PM, Jacob Maes wrote: A couple notes that may be helpful: 1. When you have a stateful processor that dies, the changelog is the default

Re: checkpoint example?

Let me explain the use case in more detail: We are keeping the data in 2 systems in sync. Let's name the upstream system as the "source" and the downstream system as "destination". The destination system is backed up (locally) once per day (let's say at 3:00 AM). It's now 1:00 PM, and we've

Re: checkpoint example?

To add to Jacob's and Jagadish's answers. If you want to to read from 24 hours before (not from the beginning or the end of the stream) you can set the checkpoint interval (see Jagadish's comment) to 24 hours. It is kind of unusual, but should work :). On Tue, Mar 1, 2016 at 4:32 PM, Jacob Maes

Re: checkpoint example?

A couple notes that may be helpful: 1. When you have a stateful processor that dies, the changelog is the default means by which the state is restored. Change logging is enabled with this config: stores.store-name.changelog 2. If, when the job comes back up, it needs to reprocess historical

Re: checkpoint example?

Users need not worry about checkpointing. Samza will automatically commit offsets every 60s. You can choose to commit more often by either 1. Setting task.commit.ms to a smaller value (or) 2. Doing manual commit yourself by setting task.commit.ms = -1. and calling taskCoordinator.commit(); I'm

Re: checkpoint example?

On 03/01/2016 03:10 PM, Jagadish Venkatraman wrote: You don't have to implement any state checkpoint. Samza automatically checkpoints state for you. When you recover from a failure/restart you will resume processing from the previous checkpoint. So, it's merely a configuration issue?

checkpoint example?

Hi folks. Is there an example or tutorial for implementing a state checkpoint?: http://samza.apache.org/learn/documentation/0.10/container/checkpointing.html Or should I just start reading the documentation? Thanks. -- Jeff Ramin Software Engineer Singlewire Software 2601 W Beltline Hwy