On 2016/02/10, 00:33, "[email protected]" <[email protected]> wrote:
>Hi, > >Then rm -rf * should not be used in any kind of file system. Why only >Lustre file system' best practices have this as a pointer. Because Lustre is more likely to have tens of thousands or millions of files in a single directory than most local filesystems. This same problem exists for local filesystems as well, for example: http://www.cyberciti.biz/faq/argument-list-too-long-error-solution/ http://stackoverflow.com/questions/11475221/why-do-i-get-bin-sh-argument-li st-too-long-when-passing-quoted-arguments Cheers, Andreas >-----Original Message----- >From: Dilger, Andreas [mailto:[email protected]] >Sent: Wednesday, February 10, 2016 11:22 AM >To: Agrawal, Prakrati PTIN-PTT/ICOE; [email protected] >Subject: Re: [lustre-discuss] Why certain commands should not be used on >Lustre file system > >On 2016/02/09, 21:16, "lustre-discuss on behalf of >[email protected]<mailto:[email protected]>" ><[email protected]<mailto:lustre-discuss-bounces@lis >ts.lustre.org> on behalf of >[email protected]<mailto:[email protected]>> wrote: > >I read on Lustre best practices that ls -U should be used instead of ls >-l . I understand that ls -l makes MDS contact all OSS to get all >information about all files and hence loads it. But, what does ls -U do >to avoid it? > > -U do not sort; list entries in directory order > >This is more important for very large directories, since "ls" will read >all of the entries and stat them before printing anything. That said, >GNU ls will still read all of the entries before printing them, so for >very large directories "find <directory> -ls" is a lot faster to start >printing entries. > >Also, it is said that rm-rf * should not be used. Please can someone >explain the reason for that. > >It is also said that instead lfs find <directory path> --type f -print0 >| xargs -0 rm -f should be used. Please explain the reason for this also. > >"rm -rf *" will expand "*" onto the command line (done by bash) and if >there are too many files in the directory (more than about 8MB IIRC) then >bash will fail to execute the command. Running "lfs find" (or just plain >"find") will only print the filenames onto the output and xargs will >process them in chunks that fit onto a command-line. > >Cheers, Andreas >-- >Andreas Dilger >Lustre Principal Architect >Intel High Performance Data Division > >_______________________________________________ >lustre-discuss mailing list >[email protected] >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel High Performance Data Division _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
