[
https://issues.apache.org/jira/browse/HDFS-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198198#comment-17198198
]
zhuqi commented on HDFS-15584:
------------------------------
Hi [~LiJinglun], in our very busy cluster with thousands of nodes, very heavy
deletion everyday causes the lock queue full for a couple of minutes. Also,
when millions of blocks are put into the pending deletion queue, NameNode will
suffer from a big performance drop. When the above situation happens, the
original block increment solution also can not solve the problem in our
cluster, so i add the patch to try to solve it.
Thanks.
> Improve HDFS large deletion cause namenode lockqueue boom and pending
> deletion boom.
> ------------------------------------------------------------------------------------
>
> Key: HDFS-15584
> URL: https://issues.apache.org/jira/browse/HDFS-15584
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.4.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
> Attachments: HDFS-15584.001.patch
>
>
> In our production cluster, the large deletion will boom the namenode lock
> queue, also will lead to the boom of pending deletion in invalidate blocks.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
