thanks. We are using kafka flink consumer 0.8.2_11 ,I have set "auto.offset.reset" to "largest" On cancel and restart the consumer is reading from where it left off instead of current offset, i tried both largest and latest in auto.offset.reset
On Wed, Aug 3, 2016 at 3:12 PM, Stephan Ewen <[email protected]> wrote: > Checkpointing starts the consumer where it left off in case the job fails > and recovers. > If you explicitly cancel a job and start a new job (same jar), the new job > will not start from a checkpoint, but from blank state. > > > On Wed, Aug 3, 2016 at 11:37 AM, Janardhan Reddy < > [email protected]> wrote: > >> I mean in case of chekpointing, won't the consumer start from where it >> previously left ? >> >> On Wed, Aug 3, 2016 at 3:06 PM, Janardhan Reddy < >> [email protected]> wrote: >> >>> How would checkpointing affect the offset. >>> >>> On Wed, Aug 3, 2016 at 3:03 PM, Stephan Ewen <[email protected]> wrote: >>> >>>> When you cancel and restart a Flink job (without a savepoint), it does >>>> not use the checkpoint data, and uses the behavior you defined in the Kafka >>>> consumer to decide where to start from (consumer group, latest, or >>>> earliest). >>>> >>>> On Wed, Aug 3, 2016 at 11:26 AM, Janardhan Reddy < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> Is there a way to read from latest offset in kafka consumer on >>>>> restart. >>>>> Or can we somehow start flink ignoring previous checkpointed data. >>>>> >>>>> Thanks >>>>> >>>> >>>> >>> >> >
