Thanks. No, it's not that big a deal now that I understand it, but as I'd had to spend a fair amount of time working out what was going on I thought I'd flag it up in case it wasn't known about.
Tim Ward -----Original Message----- From: Matthias J. Sax <matth...@confluent.io> Sent: 14 September 2018 19:57 To: users@kafka.apache.org Subject: Re: Understanding default.deserialization.exception.handler Your observation is correct. It's a known bug: https://issues.apache.org/jira/browse/KAFKA-6502 In practice, it should not be a big issue though. - you would only hit this bug if you don't process a "good message" afterwards - even if you hit this bug, you would just skip the message again Thus, the only drawback I see is an additional log message. As long as you don't have many _consecutive_ corrupted messages it should be impact you much. Hope this helps. -Matthias On 9/13/18 6:09 AM, Tim Ward wrote: > With > > > props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG, > LogAndContinueExceptionHandler.class); > > Scenario A: > > Run application. Feed a message into the topic that will fail deserialization. > Application logs exception and keeps running. > Shut down application. Restart application. > Application re-reads broken message, logs exception again (and keeps running). > > Scenario B: > > Run application. Feed a message into the topic that will fail deserialization. > Application logs exception and keeps running. > Feed a good message into deserialization. > Application processes it normally. > Shut down application. Restart application. > Application does *not* re-reads broken message. > > So it looks like LogAndContinueExceptionHandler does not seem to commit() the > incoming "poison pill" message(s), and these will be re-read if the > application is restarted without any good messages having been read after the > bad ones. > > Have I understood this correctly? If so, is this correct behaviour as > designed? Is it documented to that level of detail? > > Tim Ward > > The contents of this email and any attachment are confidential to the > intended recipient(s). If you are not an intended recipient: (i) do not use, > disclose, distribute, copy or publish this email or its contents; (ii) please > contact the sender immediately; and (iii) delete this email. Our privacy > policy is available here: https://origamienergy.com/privacy-policy/. Origami > Energy Limited (company number 8619644); Origami Storage Limited (company > number 10436515) and OSSPV001 Limited (company number 10933403), each > registered in England and each with a registered office at: Ashcombe Court, > Woolsack Way, Godalming, GU7 1LQ. > The contents of this email and any attachment are confidential to the intended recipient(s). If you are not an intended recipient: (i) do not use, disclose, distribute, copy or publish this email or its contents; (ii) please contact the sender immediately; and (iii) delete this email. Our privacy policy is available here: https://origamienergy.com/privacy-policy/. Origami Energy Limited (company number 8619644); Origami Storage Limited (company number 10436515) and OSSPV001 Limited (company number 10933403), each registered in England and each with a registered office at: Ashcombe Court, Woolsack Way, Godalming, GU7 1LQ.
