[
https://issues.apache.org/jira/browse/RATIS-2213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18107234#comment-18107234
]
Huang Kuan Hao commented on RATIS-2213:
---------------------------------------
Still reproduces on master (8f1ae8a) — added a deterministic unit test in
TestDataStreamManagement.
The DataStreamMap entry is added on STREAM_HEADER and removed only by the log
worker on link, or by StreamInfo.cleanUp. On a successful CLOSE, streams and
the channels entry are drained, but the DataStreamMap entry is left for the log
path. If the DATASTREAM log entry is never written (abort / leader change /
follower truncation) it leaks: channels no longer has the key so
channel-inactive can't reclaim it, and with a reused channel it never fires
anyway.
Patch attached: on close, after request.timeout, remove the entry iff still
unlinked (non-null remove ⇒ not linked ⇒ safe to clean). Red without the fix,
green with it; TestDataStreamManagement 12/12. Will open a PR.
> Memory leak of DataStreamMapImpl when Streaming write
> -----------------------------------------------------
>
> Key: RATIS-2213
> URL: https://issues.apache.org/jira/browse/RATIS-2213
> Project: Ratis
> Issue Type: Bug
> Components: Streaming
> Reporter: ChenXi
> Priority: Critical
> Attachments: image-2024-12-17-16-44-13-020.png,
> image-2024-12-17-17-03-19-819.png, image-2024-12-17-17-03-24-737.png
>
>
> h2. Phenomenon
> During our online test of Streaming writing, we found some memory leaks. When
> a DN write traffic exceeds 1GB/s ~ 2GB/s, Netty's Direct memory will continue
> to grow.
> h2. Version
> Ozone-1.4.1 based internal build, use the Ratis version 3.1.1
> h2. Reproduce Way
> To reproduce the DataStreamMapImpl#map and DataStreamManagement#streams leaks.
> For the detail we can refer to
> https://issues.apache.org/jira/browse/HDDS-11939
> h2. screenshot
> Direct memory increase
> !image-2024-12-17-16-44-13-020.png!
> This is a mat analysis of a memory dump of a DN long after it has stopped
> writing.
> !image-2024-12-17-17-03-19-819.png!
> !image-2024-12-17-17-03-24-737.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
