[
https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055739#comment-17055739
]
zhuqi edited comment on HDFS-15180 at 3/10/20, 9:44 AM:
--------------------------------------------------------
!image-2020-03-10-17-22-57-391.png|width=604,height=137!
Hi [~sodonnell]
It's the thread blocked avg per day monitor.
I just gray 3 nodes in busy cluster in different time , the green one is the
longest, and there are almost no blocked thread now. The two one has a good
improvement from it has beed changed to RW Lock.
And i analyze the log to find the long lock time happen when :
!image-2020-03-10-17-31-58-830.png|width=611,height=148!
DirectoryScanner scan operation.
And other does not cause too much time when very busy:
!image-2020-03-10-17-34-26-368.png|width=812,height=136!
Such as the deep copy for the caculation of dfsUsed.
was (Author: zhuqi):
!image-2020-03-10-17-22-57-391.png|width=604,height=137!
Hi [~sodonnell]
I just gray 3 nodes in busy cluster in different time , the green one is the
longest, and there are almost no blocked thread now. The two one has a good
improvement from it has beed changed to RW Lock.
And i analyze the log to find the long lock time happen when :
!image-2020-03-10-17-31-58-830.png|width=611,height=148!
DirectoryScanner scan operation.
And other does not cause too much time when very busy:
!image-2020-03-10-17-34-26-368.png|width=812,height=136!
Such as the deep copy for the caculation of dfsUsed.
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
> -----------------------------------------------------------
>
> Key: HDFS-15180
> URL: https://issues.apache.org/jira/browse/HDFS-15180
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.2.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
> Attachments: image-2020-03-10-17-22-57-391.png,
> image-2020-03-10-17-31-58-830.png, image-2020-03-10-17-34-26-368.png
>
>
> Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in
> big cluster. If we can split the FsDatasetImpl datasetLock via blockpool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
