[
https://issues.apache.org/jira/browse/KAFKA-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115684#comment-18115684
]
Ismael Juma commented on KAFKA-21101:
-------------------------------------
{quote}so dropping them loses nothing observable
{quote}
Reading out of order is problematic for some applications and they would rather
repopulate the topic from the source (if there is one). So, this bit isn't
necessarily right.
{quote}We should offer an option to drop the invalid record
{quote}
This sounds reasonable to me.
> Drop the remaining records of a batch when encountering an invalid record
> -------------------------------------------------------------------------
>
> Key: KAFKA-21101
> URL: https://issues.apache.org/jira/browse/KAFKA-21101
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: majialong
> Priority: Major
> Labels: need-kip
>
> Currently, an invalid record marks the partition "uncleanable", which blocks
> the cleaner from compacting all other records in that partition. Since the
> record is already invalid regardless of the root cause, nobody can read it or
> anything after it in the same batch (consumers parse a batch sequentially and
> fail at the same point), so dropping them loses nothing observable. We should
> offer an option to drop the invalid record and the remaining records of its
> batch, and keep the partition cleanable. Records before the invalid one are
> parsed successfully and are retained as usual.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
