[
https://issues.apache.org/jira/browse/HDFS-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199264#comment-16199264
]
Eric Payne commented on HDFS-12625:
-----------------------------------
Features like snapshots have increased the difficulty to redesign deletes to be
more asynchronous. A large delete should be profiled to target areas for
optimization. Perhaps a max limit on files that can be deleted at once may
mitigate issues.
> Reduce expense of deleting large directories
> --------------------------------------------
>
> Key: HDFS-12625
> URL: https://issues.apache.org/jira/browse/HDFS-12625
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.9.0, 2.8.1, 3.1.0
> Reporter: Eric Payne
>
> Deletion of ~5M files on a large cluster jammed the NN for 52 seconds. The
> call queue overflowed and began rejecting clients. 14k calls were queued for
> which most clients timed out while the NN was hung. Tasks issuing the calls
> likely failed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
