[
https://issues.apache.org/jira/browse/KAFKA-17499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Cook updated KAFKA-17499:
-------------------------------
Description:
There is a memory leak when using the `.all()` method of ReadOnlyKeyValueStore
(specifically MeteredTimestampedKeyValueStore).
I have recreated the problem in a small app here:
[https://github.com/jamescookie/kafka-stream-memory-leak]
When I run the test method using 3.8.0, I get this memory footprint:
!Screenshot 2024-09-09 at 08.31.57.png!
In a little over 90 seconds the number of `java.util.TreeMap$Entry` grows to
over 10 million:
!Screenshot 2024-09-09 at 08.32.16.png!
!Screenshot 2024-09-09 at 08.32.25.png!
When running against 3.7.1, this does not happen:
!Screenshot 2024-09-09 at 08.35.07.png!
The workaround (and correct way of coding) is to wrap a try-with-resources
statement around the `store.all()`.
I think this has been caused by this
[commit|https://github.com/apache/kafka/pull/15219/files] to fix KAFKA-15770
was:
There is a memory leak when using the `.all()` method of ReadOnlyKeyValueStore
(specifically MeteredTimestampedKeyValueStore).
I have recreated the problem is a small app here:
[https://github.com/jamescookie/kafka-stream-memory-leak]
When I run the test method using 3.8.0, I get this memory footprint:
!Screenshot 2024-09-09 at 08.31.57.png!
In a little over 90 seconds the number of `java.util.TreeMap$Entry` grows to
over 10 million:
!Screenshot 2024-09-09 at 08.32.16.png!
!Screenshot 2024-09-09 at 08.32.25.png!
When running against 3.7.1, this does not happen:
!Screenshot 2024-09-09 at 08.35.07.png!
The workaround (and correct way of coding) is to wrap a try-with-resources
statement around the `store.all()`.
I think this has been caused by this
[commit|https://github.com/apache/kafka/pull/15219/files] to fix KAFKA-15770
> Memory leak when using ReadOnlyKeyValueStore `all()` method
> -----------------------------------------------------------
>
> Key: KAFKA-17499
> URL: https://issues.apache.org/jira/browse/KAFKA-17499
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.8.0
> Reporter: James Cook
> Priority: Major
> Attachments: Screenshot 2024-09-09 at 08.31.57.png, Screenshot
> 2024-09-09 at 08.32.16.png, Screenshot 2024-09-09 at 08.32.25.png, Screenshot
> 2024-09-09 at 08.35.07.png
>
>
> There is a memory leak when using the `.all()` method of
> ReadOnlyKeyValueStore (specifically MeteredTimestampedKeyValueStore).
> I have recreated the problem in a small app here:
> [https://github.com/jamescookie/kafka-stream-memory-leak]
> When I run the test method using 3.8.0, I get this memory footprint:
> !Screenshot 2024-09-09 at 08.31.57.png!
> In a little over 90 seconds the number of `java.util.TreeMap$Entry` grows to
> over 10 million:
> !Screenshot 2024-09-09 at 08.32.16.png!
> !Screenshot 2024-09-09 at 08.32.25.png!
>
> When running against 3.7.1, this does not happen:
> !Screenshot 2024-09-09 at 08.35.07.png!
>
> The workaround (and correct way of coding) is to wrap a try-with-resources
> statement around the `store.all()`.
> I think this has been caused by this
> [commit|https://github.com/apache/kafka/pull/15219/files] to fix KAFKA-15770
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
