Chia-Ping Tsai created KAFKA-21100:
--------------------------------------
Summary: Defer to materialize the record body until it is
confirmed to be retained
Key: KAFKA-21100
URL: https://issues.apache.org/jira/browse/KAFKA-21100
Project: Kafka
Issue Type: Improvement
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
This is similar to KAFKA-21082.
The other path, cleanInto / filterTo, could use a similar approach to defer
loading the value until we know the record must be retained.
The only fallback is when the oversized record is the newest value of its key
and so must be kept. In this case, we could just keep the whole batch as-is
(writeOriginalBatch already does this) to avoid reading the full value. The
side effect is that batch is not compacted, but it won't violate the compaction
guarantee, since its key is already in the offset map and older values in other
batches are still cleaned. The oversized record will be dropped if it is not
the newest one :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
