[
https://issues.apache.org/jira/browse/HDDS-11939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905882#comment-17905882
]
JiangHua Zhu commented on HDDS-11939:
-------------------------------------
[~XiChen], have you enabled zero copy?
> Ratis Memory leak of DataStreamMapImpl when Stream write
> --------------------------------------------------------
>
> Key: HDDS-11939
> URL: https://issues.apache.org/jira/browse/HDDS-11939
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: ChenXi
> Priority: Critical
> Attachments: image-2024-12-15-21-14-27-612.png,
> image-2024-12-15-22-12-41-704.png, image-2024-12-15-22-14-22-904.png,
> image-2024-12-15-22-18-24-185.png, image-2024-12-15-22-21-26-187.png,
> image-2024-12-15-22-35-16-703.png
>
>
> h2. Phenomena
> We have found a memory leak in DataStreamMapImpl when writing with Stream
> online.
> This problem can be reproduced on the current master branch
> DataStreamMapImpl even after the write request stops for a long time, will
> still hold a very large number of DataStream, these DataStream unless
> restarted, or more and more, can not be released
> !image-2024-12-15-22-21-26-187.png|width=993,height=758!
> !image-2024-12-15-21-14-27-612.png|width=1209,height=474!
> h2. Reproduction method
> h3. Starting the cluster in Steaming mode
> refer to:
> [https://ozone.apache.org/docs/edge/feature/streaming-write-pipeline.html]
> h3. execute a command
>
> {code:bash}
> for i in `seq 1 10`; do ozone freon ommg --operation CREATE_STREAM_FILE -n
> 100 -t 100 --size=1M --volume s3v --bucket bucket1 --duration 5;done
> {code}
> Note: -t 100 number of client threads 100, this is the key to reproduction,
> must be multi-threaded client to reproduce the leak
> h3. h3. DataStreamMapImpl
> At this point in the DataStreamMapImpl has been left in some DataStream but
> because there is no log, so can not be directly observed,
> I added a log in the DataStreamMapImpl#remove to facilitate the observation,
> this is a screenshot of my logs
> !image-2024-12-15-22-35-16-703.png|width=851,height=382!
> !image-2024-12-15-22-18-24-185.png|width=1816,height=920!
> h4.
> h3. h3. Netty's mem
> And Netty's direct mem keeps growing, not dropping.
> Before
> !image-2024-12-15-22-12-41-704.png|width=675,height=120!
> After
> !image-2024-12-15-22-14-22-904.png|width=735,height=108!
>
>
> h3. h3. Other
> The `cleanUpOnChannelInactive` method does not clean up a leaked `DataStream`
> in `DataStreamMapImpl.`
> If you use a single-threaded client test, the leak does not occur
>
> {code:java}
> for i in `seq 1 10`; do ozone freon ommg --operation CREATE_STREAM_FILE -n
> 100 -t 1 --size=1M --volume s3v --bucket bucket1 --duration 5;done {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
