slfan1989 commented on PR #7552: URL: https://github.com/apache/ozone/pull/7552#issuecomment-2534325606
@Tejaskriya Thank you for submitting this PR! The reason I’m commenting under your PR is that I heard you’ve encountered some deletion issues. During last week’s community meeting, @ChenSammi introduced some of the deletion challenges, but we haven’t experienced the same issues on our side. I’m wondering if you could provide more details. In the past two months, we have completely transitioned our cluster from Ratis-3 replication to EC-6-3. During this process, we had a large number of deletion requests, and I’ve deleted more than 100 PB of data in these two months, with good deletion performance. I’ve summarized some experiences regarding data deletion, hoping they can help you: 1. It’s important to ensure that the ratio of OM deletion parameters to SCM deletion parameters is reasonable. These are some of the parameters we use in production. ``` -- OM ozone.path.deleting.limit.per.task 150000 ozone.directory.deleting.service.interval 180s ozone.key.deleting.limit.per.task 150000 ozone.block.deleting.service.interval 180s -- SCM hdds.scm.block.deletion.per-interval.max 2000000 hdds.scm.block.deleting.service.interval 300s ``` The `hdds.scm.block.deletion.per-interval.max` parameter should be appropriately large to ensure that the replicas of the same container can be sent to Datanodes simultaneously. 2. After we implemented HDDS-11498: Improve SCM deletion efficiency, the deletion performance saw a significant improvement. cc: @ChenSammi @adoroszlai @errose28 Additionally, I don’t recommend adding too many metrics in the DataNode, as there are already a large number of metrics, which puts considerable pressure on our collection system. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
