[
https://issues.apache.org/jira/browse/KAFKA-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115399#comment-18115399
]
Chia-Ping Tsai commented on KAFKA-21082:
----------------------------------------
another idea: The record is valid but exceeds the limit. All the cleaner needs
is the "key's hash" and "whether value is null". Hence, we could add a variant
of readPartiallyFrom which feeds the key into the digest in fixed chunks
(OffsetMap only needs the hash anyway) and skips the value, since null can be
checked by its size.
> Reuse the buffer in decompression path if the records are dropped quickly
> -------------------------------------------------------------------------
>
> Key: KAFKA-21082
> URL: https://issues.apache.org/jira/browse/KAFKA-21082
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: majialong
> Priority: Minor
>
> There are a few paths with the following pattern:
> # the records could be compressed
> # the records are dropped quickly (they never outlive the iterator)
> # the key and value are needed, so {{skipKeyValueIterator}} is not an option
> That means numerous buffers will be created for those records and then get
> dropped along with the records. That is wasteful, so we could pass a
> {{BufferSupplier}} to {{streamingIterator}} for the record bodies and release
> the buffers when the iterator is closed.
> {{Cleaner.buildOffsetMapForSegment}} has the first priority, since most
> normal records get compressed. {{RecordsIterator}} and
> {{CoordinatorLoaderImpl}} are candidates, but I'm not sure whether they get
> compressed usually 🙂
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
