On Fri, Oct 22, 2021 at 7:36 AM Helmut Jarausch <[email protected]> wrote: > > > There are more than 55,000 files on some <PREVBackUp> which is located > on a BTRFS file system. > Standard 'rm -rf' is really slow. > > Is there anything I can do about this? >
I don't have any solid suggestions as I haven't used btrfs in a while. File deletion speed is something that is very filesystem specific, but on most it tends to be slow. An obvious solution would be garbage collection, which is something used by some filesystems but I'm not aware of any mainstream ones. You can sort-of get that behavior by renaming a directory before deleting it. Suppose you have a directory created by a build system and you want to do a new build. Deleting the directory takes a long time. So, first you rename it to something else (or move it someplace on the same filesystem which is fast), then you kick off your build which no longer sees the old directory, and then you can delete the old directory slowly at your leisure. Of course, as with all garbage collection, you need to have the spare space to hold the data while it gets cleaned up. I'm not sure if btffs is any faster at deleting snapshots/reflinks than hard links. I suspect it wouldn't be, but you could test that. Instead of populating a directory with hard links, create a snapshot of the directory tree, and then rsync over it/etc. The result looks the same but is COW copies. Again, I'm not sure that btrfs will be any faster at deleting reflinks than hard links though - they're both similar metadata operations. I see there is a patch in the works for rsync that uses reflinks instead of hard links to do it all in one command. That has a lot of benefits, but again I'm not sure if it will help with deletion. You could also explore other filesystems that may or may not have faster deletion, or look to see if there is any way to optimize it on btrfs. If you can spare the space, the option of moving the directory to make it look like it was deleted will work on basically any filesystem. If you want to further automate it you could move it to a tmp directory on the same filesystem and have tmpreaper do your garbage collection. Consider using ionice to run it at a lower priority, but I'm not sure how much impact that has on metadata operations like deletion. -- Rich
