[
https://issues.apache.org/jira/browse/HDFS-12163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108359#comment-16108359
]
Weiwei Yang commented on HDFS-12163:
------------------------------------
Run 1 DN in mini ozone cluster, there were *402* threads once the cluster is
up, checked the thread dump, there are *280* RPC handler threads, with
following trace
{code}
IPC Server handler 0 on 51641@2330 - priority:5 - threadId:0x45 - nativeId:NA -
state:WAITING
stackTrace:
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:287)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2661)
{code}
this is because
{{dfs.storage.service.handler.count}} = {{10}}
{{ozone.ksm.handler.count.key}} = {{200}}
we are setting a pretty big value for KSM handlers, this is not necessary for a
mini cluster, we can overwrite this config in mini cluster to reduce the number
of threads. I will check the leak issue a bit later.
> Ozone: MiniOzoneCluster uses 400+ threads
> -----------------------------------------
>
> Key: HDFS-12163
> URL: https://issues.apache.org/jira/browse/HDFS-12163
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Weiwei Yang
> Attachments: TestOzoneThreadCount20170719.patch
>
>
> Checked the number of active threads used in MiniOzoneCluster with various
> settings:
> - Local handlers
> - Distributed handlers
> - Ratis-Netty
> - Ratis-gRPC
> The results are similar for all the settings. It uses 400+ threads for an
> 1-datanode MiniOzoneCluster.
> Moreover, there is a thread leak -- a number of the threads do not shutdown
> after the test is finished. Therefore, when tests run consecutively, the
> later tests use more threads.
> Will post the details in comments.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
