[
https://issues.apache.org/jira/browse/HDDS-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDDS-2342:
--------------------------------
Target Version/s: 0.6.0 (was: 0.5.0)
Labels: TriagePending (was: )
> ContainerStateMachine$chunkExecutor threads hold onto native memory
> -------------------------------------------------------------------
>
> Key: HDDS-2342
> URL: https://issues.apache.org/jira/browse/HDDS-2342
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Labels: TriagePending
>
> In a heap dump many threads in ContainerStateMachine$chunkExecutor holds onto
> native memory in the ThreadLocal map. Every such thread holds onto chunk
> worth of DirectByteBuffer. Since these threads are involved in write and read
> chunk operations, the JVM allocates chunk (16MB) worth of DirectByteBuffer in
> the ThreadLocalMap for every thread involved in IO. Also the native memory
> would not be GC'ed as long as the thread is alive.
> It would be better to reduce the default number of chunk executor threads and
> have them in proportion to number of disks on the datanode. We should also
> use DirectByeBuffers for the IO on datanode. Currently we allocate
> HeapByteBuffer which needs to be backed by DirectByteBuffer. If we can use a
> DirectByteBuffer we can avoid a buffer copy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
