Helo Srikanth, Thanks for reporting this, as I checked the code skippedDueToDeserializationError is effectively only recorded when the DeserializationHandlerResponse is not set to FAIL. I agree it is not exactly matching the documentation's guidance, and will try to file a JIRA and fix it.
As for skippedDueToDeserializationError and skipped-records-rate, there is an open JIRA discussing about this: https://issues.apache.org/jira/browse/KAFKA-6376, just FYI. Could you share on which version of Kafka did you observe this issue? Guozhang On Fri, Jan 26, 2018 at 6:30 AM, Srikanth <srikanth...@gmail.com> wrote: > Hello, > > As per doc when LogAndContinueExceptionHandler is used it will set > skippedDueToDeserializationError-rate metric to indicate deserialization > error. > I notice that it is never set. Instead skipped-records-rate is set. My > understanding was that skipped-records-rate is set due to timestamp > extraction errors. > > Ex, I sent a few invalid records to a topic and was able to see exception > during deserialization. > > org.apache.kafka.common.errors.SerializationException: Error deserializing > Avro message for id -1 > Caused by: org.apache.kafka.common.errors.SerializationException: Unknown > magic byte! > 18/01/24 06:50:09 WARN StreamThread: Exception caught during > Deserialization, taskId: 0_0, topic: docker.event.1, partition: 0, offset: > 3764 > > These incremented skipped-records-[rate|total]. > > Thanks, > Srikanth > -- -- Guozhang
