Thank you. Regards, Zhen Zhang Software Engineer [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> MOBILE (949) 771-6073 EMAIL zzh...@twilio.com
On Wed, Oct 7, 2020 at 11:35 PM Matthias J. Sax <mj...@apache.org> wrote: > I guess it's a question of the client implementation and you cannot > infer the behavior from the Java client. > > I would _assume_ that the offset is _not_ advanced. But the behavior is > not define by Kafka itself, but it's up the the client implementation > itself. Thus, only the client's docs or community can clarify. > > Sorry that I cannot provide a better answer. > > -Matthias > > > On 9/25/20 1:03 AM, Zhen Zhang wrote: > > Hi, > > > > Sorry for the late reply, let me clarify on this. > > > > I am developing using Golang so I used a library based on librdkafka, and > > there's one function, ReadMesage(), which is a wrapper on top of the > poll() > > function, except that it will only poll one message(record) at a time and > > return either one of the following, > > > > 1. (msg, nil) -> normal situation with no error; > > 2. (nil, err) -> err is a Kafka timeout error; > > 3. (nil, err) -> err is general error; > > 4. (msg, err) -> err is partition-specific error. > > > > When I was browsing the javadocs > > > https://kafka.apache.org/26/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html > , > > I noticed the concept of `Offsets and Consumer Position`, so I am > wondering > > that in the event of the situation 2, 3 and 4, will the position still > > increment by 1 even if what I get from ReadMessage() is an error? > > > > I tried to read the docs but didn't find anything useful I can relate > to, I > > tried to ask for help from the contributors of the library I was using > but > > didn't get an answer, not sure if this question is too obvious or too > > noobie lolol. Since that library I used is an implementation of the Kafka > > protocol, I decided to ask it here and I sincerely hope I can get some > > insights from a Kafka guru. > > > > Thanks, > > Zhen Zhang > > Software Engineer > > [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> > > MOBILE (949) 771-6073 > > EMAIL zzh...@twilio.com > > > > > > On Wed, Sep 23, 2020 at 9:45 AM Matthias J. Sax <mj...@apache.org> > wrote: > > > >> I guess it depends where the exception comes from? Can you clarify? > >> > >> -Matthias > >> > >> On 9/23/20 12:53 AM, Zhen Zhang wrote: > >>> Hi there, > >>> > >>> I am new to Kafka and I would like to get some clarifications for a > >> newbie > >>> question, > >>> > >>> Let's say if I have set up my consumer's "enable.auto.commit" to false, > >> and > >>> then poll the records one at a time. So when calling poll(), starting > >> from > >>> offset 0, if any exception is thrown, should I expect to get the record > >> at > >>> offset 0 or offset 1 when I call poll() again? The reason I'm asking > for > >>> this is bc in the Kafka Doc, it says that, > >>> "The position of the consumer gives the offset of the next record that > >> will > >>> be given out. It will be one larger than the highest offset the > consumer > >>> has seen in that partition. It automatically advances every time the > >>> consumer receives messages in a call to poll(Duration)." > >>> > >>> But in my described situation above, an exception is thrown, I'm not > sure > >>> if this is counted as a successful poll (meaning that the next poll() > >> will > >>> give the next record) or a failed one (meaning that the next poll() > will > >>> give the same record again). > >>> > >>> I would really appreciate it for your help. > >>> > >>> Thanks, > >>> Zhen Zhang > >>> Software Engineer > >>> [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> > >>> MOBILE (949) 771-6073 > >>> EMAIL zzh...@twilio.com > >>> > >> > > > >
