OK, I can use Kafka GetOffsetShell. Thanks. On Tue, Jul 14, 2015 at 10:50 AM, Yan Fang <yanfang...@gmail.com> wrote:
> Actually I have not tried this. But I do not think it will work. Because > the next offset is the offset you set + 1. There is no mechanism inside > Samza that treats -1 as the latest (though in Kafka it does). If it works, > let me know. :) I may miss something. > > Another way of doing this is to set the latest offset using the checkpoint > tool. You can get the latest offset of the topic from Kafka using > bin/kafka-run-class.sh kafka.tools.GetOffsetShell > < > https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-ConsumerOffsetChecker > > > > Thanks, > > Fang, Yan > yanfang...@gmail.com > > On Tue, Jul 14, 2015 at 10:33 AM, Bae, Jae Hyeon <metac...@gmail.com> > wrote: > > > Hi Yan > > > > Thanks for your response. I want to reset the offset just once now. If I > > use systems.system-name.streams.stream-name.samza.reset.offset in the > > configuration file, before restarting the job, I need to remove those > > properties again. That's why I don't want to use that property. > > > > Is it OK setting invalid offset such as -1 or Long.MAX_VALUE? > > > > On Tue, Jul 14, 2015 at 10:16 AM, Yan Fang <yanfang...@gmail.com> wrote: > > > > > Hi Jae, > > > > > > Do you want to reset the offset to the latest one when you *start *the > > job > > > or when the just is *running*? > > > > > > If it's the former one, you can use > > > > > > systems.system-name.samza.offset.default=upcoming > > > systems.system-name.streams.stream-name.samza.reset.offset=true > > > > > > What is the reason that you do not want to use > > > systems.system-name.streams.stream-name.samza.reset.offset > > > ? > > > > > > Thanks, > > > > > > Fang, Yan > > > yanfang...@gmail.com > > > > > > On Tue, Jul 14, 2015 at 12:49 AM, Bae, Jae Hyeon <metac...@gmail.com> > > > wrote: > > > > > > > Hi > > > > > > > > I want to reset the offset for the job to the latest one, which > means I > > > > want to ignore them without using > > > > systems.system-name.streams.stream-name.samza.reset.offset option. > > > > > > > > If I use checkpoint tool and reset the offset as -1 or > Long.MAX_VALUE, > > in > > > > my theory, kafka consumer will throw an exception and it will reset > > them > > > as > > > > the latest one, am I right? Otherwise, please let me know how to > reset > > > > offsets to the latest one. > > > > > > > > Thank you > > > > Best, Jae > > > > > > > > > >
