[
https://issues.apache.org/jira/browse/KAFKA-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated KAFKA-21100:
-----------------------------------
Summary: Defer materializing the record body until the record is confirmed
to be retained (was: Defer to materialize the record body until it is
confirmed to be retained)
> Defer materializing the record body until the record is confirmed to be
> retained
> --------------------------------------------------------------------------------
>
> Key: KAFKA-21100
> URL: https://issues.apache.org/jira/browse/KAFKA-21100
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: majialong
> Priority: Major
>
> This is similar to KAFKA-21082.
> The other path, cleanInto / filterTo, could use a similar approach to defer
> loading the value until we know the record must be retained.
> The only fallback is when the oversized record is the newest value of its key
> and so must be kept. In this case, we could just keep the whole batch as-is
> (writeOriginalBatch already does this) to avoid reading the full value. The
> side effect is that batch is not compacted, but it won't violate the
> compaction guarantee, since its key is already in the offset map and older
> values in other batches are still cleaned. The oversized record will be
> dropped if it is not the newest one :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
