I guess it's a question of the client implementation and you cannot infer the behavior from the Java client.
I would _assume_ that the offset is _not_ advanced. But the behavior is not define by Kafka itself, but it's up the the client implementation itself. Thus, only the client's docs or community can clarify. Sorry that I cannot provide a better answer. -Matthias On 9/25/20 1:03 AM, Zhen Zhang wrote: > Hi, > > Sorry for the late reply, let me clarify on this. > > I am developing using Golang so I used a library based on librdkafka, and > there's one function, ReadMesage(), which is a wrapper on top of the poll() > function, except that it will only poll one message(record) at a time and > return either one of the following, > > 1. (msg, nil) -> normal situation with no error; > 2. (nil, err) -> err is a Kafka timeout error; > 3. (nil, err) -> err is general error; > 4. (msg, err) -> err is partition-specific error. > > When I was browsing the javadocs > https://kafka.apache.org/26/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html, > I noticed the concept of `Offsets and Consumer Position`, so I am wondering > that in the event of the situation 2, 3 and 4, will the position still > increment by 1 even if what I get from ReadMessage() is an error? > > I tried to read the docs but didn't find anything useful I can relate to, I > tried to ask for help from the contributors of the library I was using but > didn't get an answer, not sure if this question is too obvious or too > noobie lolol. Since that library I used is an implementation of the Kafka > protocol, I decided to ask it here and I sincerely hope I can get some > insights from a Kafka guru. > > Thanks, > Zhen Zhang > Software Engineer > [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> > MOBILE (949) 771-6073 > EMAIL zzh...@twilio.com > > > On Wed, Sep 23, 2020 at 9:45 AM Matthias J. Sax <mj...@apache.org> wrote: > >> I guess it depends where the exception comes from? Can you clarify? >> >> -Matthias >> >> On 9/23/20 12:53 AM, Zhen Zhang wrote: >>> Hi there, >>> >>> I am new to Kafka and I would like to get some clarifications for a >> newbie >>> question, >>> >>> Let's say if I have set up my consumer's "enable.auto.commit" to false, >> and >>> then poll the records one at a time. So when calling poll(), starting >> from >>> offset 0, if any exception is thrown, should I expect to get the record >> at >>> offset 0 or offset 1 when I call poll() again? The reason I'm asking for >>> this is bc in the Kafka Doc, it says that, >>> "The position of the consumer gives the offset of the next record that >> will >>> be given out. It will be one larger than the highest offset the consumer >>> has seen in that partition. It automatically advances every time the >>> consumer receives messages in a call to poll(Duration)." >>> >>> But in my described situation above, an exception is thrown, I'm not sure >>> if this is counted as a successful poll (meaning that the next poll() >> will >>> give the next record) or a failed one (meaning that the next poll() will >>> give the same record again). >>> >>> I would really appreciate it for your help. >>> >>> Thanks, >>> Zhen Zhang >>> Software Engineer >>> [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> >>> MOBILE (949) 771-6073 >>> EMAIL zzh...@twilio.com >>> >> >
signature.asc
Description: OpenPGP digital signature
