[
https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055798#comment-17055798
]
Stephen O'Donnell commented on HDFS-15180:
------------------------------------------
[~zhuqi] Thanks for sharing this, it looks promising. It is also good to see
the patch used in a real world cluster without any issues.
Looking at your chart, are the orange and blue lines using the old code until
around 5th / 6th March, then you switched to the new RW Fair lock plus
HDFS-15160 and the blocked thread count has reduced to almost zero? The green
line has been using HDFS-15160 since at least 4th March?
I am not sure what metrics we should track to prove this change it good, but
blocked thread count seems like a good one for now. Your chart looks promising.
It would also be good to see a line on your chart for 2 or 3 nodes where
HDFS-15160 is NOT applied for a comparison over time, so we can clearly see the
nodes with HDFS-15160 against nodes without it.
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
> -----------------------------------------------------------
>
> Key: HDFS-15180
> URL: https://issues.apache.org/jira/browse/HDFS-15180
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.2.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
> Attachments: image-2020-03-10-17-22-57-391.png,
> image-2020-03-10-17-31-58-830.png, image-2020-03-10-17-34-26-368.png
>
>
> Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in
> big cluster. If we can split the FsDatasetImpl datasetLock via blockpool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
