We actually use
find <dir> -type f -print0 | xargs -n 100 -P 32 -0 -- rm -f
which will parallelise the rm... which runs a fair bit faster.
On Wed, Feb 10, 2016 at 3:33 PM, <[email protected]> wrote:
> Hi,
>
> Then rm -rf * should not be used in any kind of file system. Why only Lustre
> file system' best practices have this as a pointer.
>
> Thanks and Regards,
> Prakrati
>
> -----Original Message-----
> From: Dilger, Andreas [mailto:[email protected]]
> Sent: Wednesday, February 10, 2016 11:22 AM
> To: Agrawal, Prakrati PTIN-PTT/ICOE; [email protected]
> Subject: Re: [lustre-discuss] Why certain commands should not be used on
> Lustre file system
>
> On 2016/02/09, 21:16, "lustre-discuss on behalf of
> [email protected]<mailto:[email protected]>"
> <[email protected]<mailto:[email protected]>
> on behalf of [email protected]<mailto:[email protected]>>
> wrote:
>
> I read on Lustre best practices that ls -U should be used instead of ls -l .
> I understand that ls -l makes MDS contact all OSS to get all information
> about all files and hence loads it. But, what does ls -U do to avoid it?
>
> -U do not sort; list entries in directory order
>
> This is more important for very large directories, since "ls" will read all
> of the entries and stat them before printing anything. That said, GNU ls
> will still read all of the entries before printing them, so for very large
> directories "find <directory> -ls" is a lot faster to start printing entries.
>
> Also, it is said that rm-rf * should not be used. Please can someone explain
> the reason for that.
>
> It is also said that instead lfs find <directory path> --type f -print0 |
> xargs -0 rm -f should be used. Please explain the reason for this also.
>
> "rm -rf *" will expand "*" onto the command line (done by bash) and if there
> are too many files in the directory (more than about 8MB IIRC) then bash will
> fail to execute the command. Running "lfs find" (or just plain "find") will
> only print the filenames onto the output and xargs will process them in
> chunks that fit onto a command-line.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel High Performance Data Division
>
> _______________________________________________
> lustre-discuss mailing list
> [email protected]
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
--
Dr Stuart Midgley
[email protected]
_______________________________________________
lustre-discuss mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
