[
https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204434#comment-17204434
]
Xiaoqiao He commented on HDFS-15382:
------------------------------------
Thanks [~sodonnell],[~Jiang Xin] for your comments.
HDFS-15150 and HDFS-15160 is very interesting improvement for DataNode, and I
think the result is also impressive. But this solution does not solve coupling
issue between different BlockPools and different Volumes when enable Federation
feature. Especially one of BlockPool/Volume's load is very high, other
BlockPools/Volumes read/write operation will be blocked since still some IO
operation in Lock which could hold for long time, such as
#updateReplicaUnderRecovery. In our inner branch, this issue is very critical.
Please reference
https://drive.google.com/file/d/1eaE8vSEhIli0H3j2eDiPJNYuKAC0MFgu/view?usp=sharing
if interesting details.
IMO, the key of this improvement is decoupling BlockPools and Volumes and try
to improve performance further. with HDFS-15150 and HDFS-15160, it will get a
better result.
About the demo patch, if agreement, we will split to some subtask to push this
feature forwards. cc [~Aiphag0]
Thanks [~sodonnell] and [~LiJinglun] again. Welcome more discussion and
suggestions.
> Split FsDatasetImpl from blockpool lock to blockpool volume lock
> -----------------------------------------------------------------
>
> Key: HDFS-15382
> URL: https://issues.apache.org/jira/browse/HDFS-15382
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Aiphago
> Assignee: Aiphago
> Priority: Major
> Attachments: HDFS-15382-sample.patch, image-2020-06-02-1.png,
> image-2020-06-03-1.png
>
>
> In HDFS-15180 we split lock to blockpool grain size.But when one volume is in
> heavy load and will block other request which in same blockpool but different
> volume.So we split lock to two leval to avoid this happend.And to improve
> datanode performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
