[
https://issues.apache.org/jira/browse/KAFKA-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated KAFKA-21082:
-----------------------------------
Description:
-There are a few paths with the following pattern:-
# -the records could be compressed-
# -the records are dropped quickly (they never outlive the iterator)-
# -the key and value are needed, so {{skipKeyValueIterator}} is not an option-
-That means numerous buffers will be created for those records and then get
dropped along with the records. That is wasteful, so we could pass a
{{BufferSupplier}} to {{streamingIterator}} for the record bodies and release
the buffers when the iterator is closed.-
-{{Cleaner.buildOffsetMapForSegment}} has the first priority, since most normal
records get compressed. {{RecordsIterator}} and {{CoordinatorLoaderImpl}} are
candidates, but I'm not sure whether they get compressed usually 🙂-
was:
There are a few paths with the following pattern:
# the records could be compressed
# the records are dropped quickly (they never outlive the iterator)
# the key and value are needed, so {{skipKeyValueIterator}} is not an option
That means numerous buffers will be created for those records and then get
dropped along with the records. That is wasteful, so we could pass a
{{BufferSupplier}} to {{streamingIterator}} for the record bodies and release
the buffers when the iterator is closed.
{{Cleaner.buildOffsetMapForSegment}} has the first priority, since most normal
records get compressed. {{RecordsIterator}} and {{CoordinatorLoaderImpl}} are
candidates, but I'm not sure whether they get compressed usually 🙂
> Reuse the buffer in decompression path if the records are dropped quickly
> -------------------------------------------------------------------------
>
> Key: KAFKA-21082
> URL: https://issues.apache.org/jira/browse/KAFKA-21082
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: majialong
> Priority: Minor
>
> -There are a few paths with the following pattern:-
> # -the records could be compressed-
> # -the records are dropped quickly (they never outlive the iterator)-
> # -the key and value are needed, so {{skipKeyValueIterator}} is not an
> option-
> -That means numerous buffers will be created for those records and then get
> dropped along with the records. That is wasteful, so we could pass a
> {{BufferSupplier}} to {{streamingIterator}} for the record bodies and release
> the buffers when the iterator is closed.-
> -{{Cleaner.buildOffsetMapForSegment}} has the first priority, since most
> normal records get compressed. {{RecordsIterator}} and
> {{CoordinatorLoaderImpl}} are candidates, but I'm not sure whether they get
> compressed usually 🙂-
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
