[
https://issues.apache.org/jira/browse/HDFS-16600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555890#comment-17555890
]
ZanderXu edited comment on HDFS-16600 at 6/18/22 10:55 AM:
-----------------------------------------------------------
Thanks [~hexiaoqiao]. The Fine-Grained locking we adopted is slightly different
from this features, but we also split the global lock into BlockPoolSlice, and
also has very obvious performance benefits.
Recently we plan to patch this features into Hadoop-2.10 and deploy on my prod
cluster which has about 1w+ DNs. After a period of deployment, I will provide
the detail performance result.
was (Author: xuzq_zander):
Thanks [~hexiaoqiao]. The Fine-Grained locking we adopted is slightly different
from this features, but we also split the global lock into BlockPoolSlice, and
also has very obvious performance benefits.
Recently we plan to patch this features into Hadoop-2.10 and deploy on my prod
cluster which has about 7000+ DNs. After a period of deployment, I will provide
the detail performance result.
> Fix deadlock of fine-grain lock for FsDatastImpl of DataNode.
> -------------------------------------------------------------
>
> Key: HDFS-16600
> URL: https://issues.apache.org/jira/browse/HDFS-16600
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> The UT
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.testSynchronousEviction
> failed, because happened deadlock, which is introduced by
> [HDFS-16534|https://issues.apache.org/jira/browse/HDFS-16534].
> DeadLock:
> {code:java}
> // org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.createRbw line 1588
> need a read lock
> try (AutoCloseableLock lock = lockManager.readLock(LockLevel.BLOCK_POOl,
> b.getBlockPoolId()))
> // org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.evictBlocks line
> 3526 need a write lock
> try (AutoCloseableLock lock = lockManager.writeLock(LockLevel.BLOCK_POOl,
> bpid))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
