[
https://issues.apache.org/jira/browse/RATIS-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899062#comment-17899062
]
Ivan Andika commented on RATIS-2189:
------------------------------------
[~szetszwo] Thanks for the feedbacks.
> please see if the OutOfMemoryError can be fixed by this. I look forward to
> hearing about your benchmark results.
Unfortunately, even when we applied the patch and disable the direct buffer.
The direct memory usage still seemed high.
> HDDS-9070 has introduced a new Netty memory consumption metrics (thanks
> [~duongnguyen] for the info). We could see if Netty has used a lot of memory.
Yes, we used this metrics to show the high Netty used direct memory and it
shows that the used direct memory usage is still high when we send a lot of
writes.
{code:java}
{
"name" : "Hadoop:service=HddsDatanode,name=NettyMetrics",
"modelerType" : "NettyMetrics",
"tag.Context" : "Netty metrics",
"tag.Hostname" : "REDACTED",
"USED_DIRECT_MEM" : 16735273239,
"MAX_DIRECT_MEM" : 16750411776
} {code}
> Use ByteBufAllocator#ioBuffer in NettyDataStreamUtils
> -----------------------------------------------------
>
> Key: RATIS-2189
> URL: https://issues.apache.org/jira/browse/RATIS-2189
> Project: Ratis
> Issue Type: Improvement
> Components: Streaming
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Minor
> Fix For: 3.2.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, NettyDataStreamUtils uses ByteBufAllocator#directBuffer which
> forces all ByteBufAllocator to allocate direct buffer even for
> PreferHeapByteBufAllocator (e.g. when we set
> -Dorg.apache.ratis.thirdparty.io.netty.noPreferDirect=true).
> It's better to use ioBuffer and delegates to the actual ByteBufAllocator to
> the type of memory it will use.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
