[ https://issues.apache.org/jira/browse/KAFKA-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-6376: ----------------------------------- Labels: needs-kip (was: ) > Improve Streams metrics for skipped recrods > ------------------------------------------- > > Key: KAFKA-6376 > URL: https://issues.apache.org/jira/browse/KAFKA-6376 > Project: Kafka > Issue Type: Bug > Components: metrics, streams > Affects Versions: 1.0.0 > Reporter: Matthias J. Sax > Labels: needs-kip > > Copy this from KIP-210 discussion thread: > {quote} > Note that currently we have two metrics for `skipped-records` on different > levels: > 1) on the highest level, the thread-level, we have a `skipped-records`, > that records all the skipped records due to deserialization errors. > 2) on the lower processor-node level, we have a > `skippedDueToDeserializationError`, that records the skipped records on > that specific source node due to deserialization errors. > So you can see that 1) does not cover any other scenarios and can just be > thought of as an aggregate of 2) across all the tasks' source nodes. > However, there are other places that can cause a record to be dropped, for > example: > 1) https://issues.apache.org/jira/browse/KAFKA-5784: records could be > dropped due to window elapsed. > 2) KIP-210: records could be dropped on the producer side. > 3) records could be dropped during user-customized processing on errors. > {quote} > [~guozhang] Not sure what you mean by "3) records could be dropped during > user-customized processing on errors." > Btw: we also drop record with {{null}} key and/or value for certain DSL > operations. This should be included as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)