Re: 'rm' Can not delete files

[Karl Vogel]

>> On Fri, 10 Feb 2012 10:34:20 -0500, 
>> Henry Olyer  said:

H> I never learned a shell language.  I suppose no one is as dumb as
H> someone who choose's not to learn, so, what's the right one.  csh?

   Not for scripting:
   http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

   I work in an environment containing FreeBSD, Solaris, and Linux boxes.
   Using a subset of BASH that also runs under the Korn Shell helps avoid
   portability surprises if I want to get something running on a different
   box, or if I post a script to a mailing list.

   If you're already in the habit of writing BASH and you have /bin/ksh
   installed, running "ksh -n" will syntax-check your script without
   executing it.  I *think* pdksh does the same thing.

-- 
Karl Vogel  I don't speak for the USAF or my company

My choices in life were either to be a piano player in a whore-house or
a politician.  And to tell the truth, there's hardly any difference.
--Harry Truman
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Daniel Staal]

--As of February 10, 2012 4:24:58 PM +, Matthew Seaman is alleged to 
have said:



On 10/02/2012 16:04, Matthew Story wrote:

find . -type f -depth 1 -print0 | xargs -n99 -0 -s8192 -c5 rm --

or some such, depending on your needs, I believe in most situations this
particular invocation will also out-perform find ... -delete.


Why would you believe that? find ... -delete calls unlink(2) directly on
each file it finds as it searches the directory tree given that it
matches the other find predicates.

Whereas find ... -print0 | xargs ... rm ... involves a whole complicated
sequence of find doing the same searching and matching job, then
marshalling lists of filenames, piping them between processes, then
xargs exec(2)ing rm with chunks of that arglist; each rm invocation then
finally ... calling unlink(2) on each of the named files.


On the other hand, passing it through xargs makes the deletion 
multi-threaded (well, multi-process), while using -delete keeps it all in 
one process.  Depending on your execution environment, that may be a win.



Actually, I doubt you'ld see much difference above the noise in the
speed of either of those two commands: they're both going to spend the
vast majority of the time waiting for disk IO, and that's common to any
way of doing this job.


This is likely the root of the issue however.  ;)   (There are probably 
some pathological cases of multi-processor, multi-controller, multi-disk 
systems where having multiple IO streams would make a difference, but they 
are likely to be few for something like this.)


Daniel T. Staal

---
This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[RW]

On Sat, 11 Feb 2012 12:05:14 +1100
andrew clarke wrote:

> On Fri 2012-02-10 16:12:06 UTC+, Matthew Seaman
> (m.sea...@infracaninophile.co.uk) wrote:
> 
> > > In addition, I don't believe it solves the OP's initial problem
> > > of the argument list being too long!  You'd probably need to use
> > > the xargs -n switch here.
> > 
> > Go and read the xargs(1) man page carefully.  xargs is specifically
> > designed to avoid arglist overflows.
> 
> Ah, I grepped for 'limit' and 'overflow', didn't see anything
> applicable, and didn't notice the -s switch.  That it avoids arglist
> overflows should perhaps be written more obviously in the man page
> (though I'm not sure how...)


The important passage is the description of what xargs does:

"Any arguments specified on the command line are given to utility
 upon each invocation, followed by some number of the arguments read
 from the standard input of xargs.  This is repeated until standard
 input is exhausted."

It would be extremely perverse to go to the trouble of breaking-up
stdin into chunks, but to choose a size that's too big to pass to the
utility. 

You expect a man page to document perversity, but not to document all
the perverse thing that aren't done.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Matthew Seaman]

On 11/02/2012 15:33, per...@pluto.rain.com wrote:
> Matthew Seaman  wrote:
> 
> ls -1 | xargs rm
>>
 but be aware that that wont work for filenames with spaces.
>>
>> True.  Can't do that using ls to generate the list of filenames as
>> there is no option to generate a null-separated list amongst ls's
>> multitudinous collection.
> 
> It can, however, be done indirectly :)
> 
> $ ls -1 | tr '\012' '\000' | xargs -0 rm

Until you run into someone who creates filenames with newlines in them...

Well, actually in that case, ls(1) would usually mangle the name:

$ touch 'a
> file name with
> new lines in it'
$ ls a*
a?file name with?new lines in it

Since it uses '?' to replace any "unprintable" character, that's not
bullet proof either.  Of course, ls(1) has an option for that:

$ ls -B a*
a\012file name with\012new lines in it

Actually, it has two options for that:

$ ls -b a*
a\nfile name with\nnew lines in it

and before Randal picks me up again on the differences between ls(1)
outputting to the terminal rather than into anything else...

$ ls a* > ls.out
$ hd ls.out 
  61 0a 66 69 6c 65 20 6e  61 6d 65 20 77 69 74 68  |a.file name
with|
0010  0a 6e 65 77 20 6c 69 6e  65 73 20 69 6e 20 69 74  |.new lines
in it|
*
0020

No mangling occurs if you pipe the output into another process.

Even so, the conclusion must be that ls(1) is not the best choice for
generating a list of file names *for programmatic input* if you have to
deal with the full gamut of possible names.  find(1) is much better in
that case.  ls(1) is great for displaying to people in pretty much all
circumstances, and it should only be used programmatically if you can
/guarantee/ you aren't going run into any of the tricky cases.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: 'rm' Can not delete files

[perryh]

Matthew Seaman  wrote:

> >>> ls -1 | xargs rm
>
> >> but be aware that that wont work for filenames with spaces.
>
> True.  Can't do that using ls to generate the list of filenames as
> there is no option to generate a null-separated list amongst ls's
> multitudinous collection.

It can, however, be done indirectly :)

$ ls -1 | tr '\012' '\000' | xargs -0 rm
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[perryh]

Jerry McAllister  wrote:

> On Fri, Feb 10, 2012 at 10:34:20AM -0500, Henry Olyer wrote:
> > I use bash 4.
>
> OK.  So??

If you had read the thread before posting, you would have known
that someone asked which shell Henry was using (and he answered).
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[andrew clarke]

On Fri 2012-02-10 16:12:06 UTC+, Matthew Seaman 
(m.sea...@infracaninophile.co.uk) wrote:

> > In addition, I don't believe it solves the OP's initial problem of the
> > argument list being too long!  You'd probably need to use the xargs -n
> > switch here.
> 
> Go and read the xargs(1) man page carefully.  xargs is specifically
> designed to avoid arglist overflows.

Ah, I grepped for 'limit' and 'overflow', didn't see anything
applicable, and didn't notice the -s switch.  That it avoids arglist
overflows should perhaps be written more obviously in the man page
(though I'm not sure how...)

> >> Or the scenic route, using xargs, with one rm per file (slower):
> >>
> >> find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f
> >>
> >> (The "scenic route" is useful if you want to do something else with
> >> the files instead of deleting them with rm.)
> 
> In this case, if you're going to call rm repeatedly with only one arg,
> then xargs is pretty pointless.  You might as well do:
> 
>find . -type f -depth 1 -exec rm -f '{}' ';'
> 
> but let's not leave people in any doubt that this is not the best option.

True, but I can never remember the syntax for -exec.  :-)

Regards
Andrew
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Randal L. Schwartz]

> "Matthew" == Matthew Seaman  writes:

Matthew>find . -type f -depth 1 -exec rm -f '{}' ';'

Matthew> but let's not leave people in any doubt that this is not the
Matthew> best option.

However...

  find . -type f -depth 1 -exec rm -f {} +

Might very well be a great option.  Well, not for something that is also
like -delete, but for other things that would have formerly required
"-print0 | xargs -0"... this is apparently a fairly recent (and
welcome!) modification to find.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
 http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Matthew Seaman]

On 10/02/2012 16:04, Matthew Story wrote:
> find . -type f -depth 1 -print0 | xargs -n99 -0 -s8192 -c5 rm --
> 
> or some such, depending on your needs, I believe in most situations this
> particular invocation will also out-perform find ... -delete.

Why would you believe that? find ... -delete calls unlink(2) directly on
each file it finds as it searches the directory tree given that it
matches the other find predicates.

Whereas find ... -print0 | xargs ... rm ... involves a whole complicated
sequence of find doing the same searching and matching job, then
marshalling lists of filenames, piping them between processes, then
xargs exec(2)ing rm with chunks of that arglist; each rm invocation then
finally ... calling unlink(2) on each of the named files.

Actually, I doubt you'ld see much difference above the noise in the
speed of either of those two commands: they're both going to spend the
vast majority of the time waiting for disk IO, and that's common to any
way of doing this job.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: 'rm' Can not delete files

[Jerry McAllister]

On Fri, Feb 10, 2012 at 10:34:20AM -0500, Henry Olyer wrote:

> So what do I change if I want to increase the shell's file limit?

You don't want to diddle the shell. 
Use the correct UNIX utilities such as - for, xargs or find - in this 
case as have been suggested by other responders.   That is the way it
is done (and done better) in UNIX.

> 
> I use bash 4.

OK.  So??

> I never learned a shell language.  I suppose no one is as dumb as someone
> who choose's not to learn, so, what's the right one.  csh?, because I do a
> lot of scientific work?, or should I be looking at another?
> 

Probably doesn't matter that much what type of work you are
doing.  It matters more what the users of a system tend to use.

In FreeBSD it is  tcsh  which is basically an extension of csh.
Actually, nowdays on FreeBSD, csh is just a link to tcsh anyway.

On Lunix, I think most people use bash, so if you are there, do that.

There are some small differences that are meaningful depending on 
what kind of scripting you are doing.   But, mostly it doesn't matter
a lot.

jerry

> 
> 
> On Wed, Feb 8, 2012 at 10:25 PM, andrew clarke  wrote:
> 
> > On Tue 2012-02-07 23:17:16 UTC+, RW (rwmailli...@googlemail.com)
> > wrote:
> >
> > > On Tue, 07 Feb 2012 22:14:56 +
> > > Matthew Seaman wrote:
> > >
> > > > ls -1 | xargs rm
> > >
> > > but be aware that that wont work for filenames with spaces.
> >
> > In addition, I don't believe it solves the OP's initial problem of the
> > argument list being too long!  You'd probably need to use the xargs -n
> > switch here.
> >
> > The above will also try to 'rm' directories, which won't work.
> >
> > Instead I would use 'find':
> >
> > find . -type f -depth 1 -delete
> >
> > This will also work with filenames with spaces.
> >
> > Or the scenic route, using xargs, with one rm per file (slower):
> >
> > find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f
> >
> > (The "scenic route" is useful if you want to do something else with
> > the files instead of deleting them with rm.)
> >
> > Regards
> > Andrew
> > ___
> > freebsd-questions@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > To unsubscribe, send any mail to "
> > freebsd-questions-unsubscr...@freebsd.org"
> >
> ___
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Matthew Seaman]

>>> ls -1 | xargs rm

>> but be aware that that wont work for filenames with spaces.

True.  Can't do that using ls to generate the list of filenames as there
is no option to generate a null-separated list amongst ls's
multitudinous collection.

> In addition, I don't believe it solves the OP's initial problem of the
> argument list being too long!  You'd probably need to use the xargs -n
> switch here.

Go and read the xargs(1) man page carefully.  xargs is specifically
designed to avoid arglist overflows.

>> Or the scenic route, using xargs, with one rm per file (slower):
>>
>> find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f
>>
>> (The "scenic route" is useful if you want to do something else with
>> the files instead of deleting them with rm.)

In this case, if you're going to call rm repeatedly with only one arg,
then xargs is pretty pointless.  You might as well do:

   find . -type f -depth 1 -exec rm -f '{}' ';'

but let's not leave people in any doubt that this is not the best option.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: 'rm' Can not delete files

[Matthew Story]

On Fri, Feb 10, 2012 at 11:04 AM, Matthew Story wrote:

> On Fri, Feb 10, 2012 at 10:51 AM, Da Rock <
> freebsd-questi...@herveybayaustralia.com.au> wrote:
>
>> On 02/11/12 01:34, Henry Olyer wrote:
>>
>>> So what do I change if I want to increase the shell's file limit?
>>>
>> I don't think you can. It's not a shell limit. It's a limit to the number
>> of arguments the command itself will take. As said, the shell expands '*'
>> to a list of files as the argument, and rm is limited to the number of
>> arguments it will parse.
>>
>>  I use bash 4.
>>>
>>> And by the way, for me, part of the normal installation of a new FBSD box
>>> is to make certain changes.  For example, for "uniq -c" I use "%06"
>>> instead
>>> of "%d" because this way I can sort the output.  Things like that.
>>>
>>> I never learned a shell language.  I suppose no one is as dumb as someone
>>> who choose's not to learn, so, what's the right one.  csh?, because I do
>>> a
>>> lot of scientific work?, or should I be looking at another?
>>>
>> There's not really much difference in this factor for shell types; as for
>> changes you'd have to hack the command's (say rm) code.
>>
>> As mentioned, I'd use the find -delete combination.
>
>
> I think the only thing that would give you this sort of pseudo-granularity
> of MAX_ARGS (and ARG_MAX) control at run-time is xargs with the -s and -n
> options ... a play on andrew's earlier example:
>
>
find . -type f -depth 1 -print0 | xargs -n99 -0 -s8192 -c5 rm --
>

the -c5 here should read -P5 ... apologies.


> or some such, depending on your needs, I believe in most situations this
> particular invocation will also out-perform find ... -delete.
>
>
>>  On Wed, Feb 8, 2012 at 10:25 PM, andrew clarke
>>>  wrote:
>>>
>>>  On Tue 2012-02-07 23:17:16 UTC+, RW (rwmailli...@googlemail.com)
 wrote:

  On Tue, 07 Feb 2012 22:14:56 +
> Matthew Seaman wrote:
>
>  ls -1 | xargs rm
>>
> but be aware that that wont work for filenames with spaces.
>
 In addition, I don't believe it solves the OP's initial problem of the
 argument list being too long!  You'd probably need to use the xargs -n
 switch here.

 The above will also try to 'rm' directories, which won't work.

 Instead I would use 'find':

 find . -type f -depth 1 -delete

 This will also work with filenames with spaces.

 Or the scenic route, using xargs, with one rm per file (slower):

 find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f

 (The "scenic route" is useful if you want to do something else with
 the files instead of deleting them with rm.)

 Regards
 Andrew
 __**_
 freebsd-questions@freebsd.org mailing list
 http://lists.freebsd.org/**mailman/listinfo/freebsd-**questions
 To unsubscribe, send any mail to "
 freebsd-questions-unsubscribe@**freebsd.org
 "

  __**_
>>> freebsd-questions@freebsd.org mailing list
>>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**questions
>>> To unsubscribe, send any mail to "freebsd-questions-**
>>> unsubscr...@freebsd.org "
>>>
>>
>> __**_
>> freebsd-questions@freebsd.org mailing list
>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**questions
>> To unsubscribe, send any mail to "freebsd-questions-**
>> unsubscr...@freebsd.org "
>>
>
>
>
> --
> regards,
> matt
>



-- 
regards,
matt
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Matthew Story]

On Fri, Feb 10, 2012 at 10:51 AM, Da Rock <
freebsd-questi...@herveybayaustralia.com.au> wrote:

> On 02/11/12 01:34, Henry Olyer wrote:
>
>> So what do I change if I want to increase the shell's file limit?
>>
> I don't think you can. It's not a shell limit. It's a limit to the number
> of arguments the command itself will take. As said, the shell expands '*'
> to a list of files as the argument, and rm is limited to the number of
> arguments it will parse.
>
>  I use bash 4.
>>
>> And by the way, for me, part of the normal installation of a new FBSD box
>> is to make certain changes.  For example, for "uniq -c" I use "%06"
>> instead
>> of "%d" because this way I can sort the output.  Things like that.
>>
>> I never learned a shell language.  I suppose no one is as dumb as someone
>> who choose's not to learn, so, what's the right one.  csh?, because I do a
>> lot of scientific work?, or should I be looking at another?
>>
> There's not really much difference in this factor for shell types; as for
> changes you'd have to hack the command's (say rm) code.
>
> As mentioned, I'd use the find -delete combination.


I think the only thing that would give you this sort of pseudo-granularity
of MAX_ARGS (and ARG_MAX) control at run-time is xargs with the -s and -n
options ... a play on andrew's earlier example:

find . -type f -depth 1 -print0 | xargs -n99 -0 -s8192 -c5 rm --

or some such, depending on your needs, I believe in most situations this
particular invocation will also out-perform find ... -delete.


>  On Wed, Feb 8, 2012 at 10:25 PM, andrew clarke  wrote:
>>
>>  On Tue 2012-02-07 23:17:16 UTC+, RW (rwmailli...@googlemail.com)
>>> wrote:
>>>
>>>  On Tue, 07 Feb 2012 22:14:56 +
 Matthew Seaman wrote:

  ls -1 | xargs rm
>
 but be aware that that wont work for filenames with spaces.

>>> In addition, I don't believe it solves the OP's initial problem of the
>>> argument list being too long!  You'd probably need to use the xargs -n
>>> switch here.
>>>
>>> The above will also try to 'rm' directories, which won't work.
>>>
>>> Instead I would use 'find':
>>>
>>> find . -type f -depth 1 -delete
>>>
>>> This will also work with filenames with spaces.
>>>
>>> Or the scenic route, using xargs, with one rm per file (slower):
>>>
>>> find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f
>>>
>>> (The "scenic route" is useful if you want to do something else with
>>> the files instead of deleting them with rm.)
>>>
>>> Regards
>>> Andrew
>>> __**_
>>> freebsd-questions@freebsd.org mailing list
>>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**questions
>>> To unsubscribe, send any mail to "
>>> freebsd-questions-unsubscribe@**freebsd.org
>>> "
>>>
>>>  __**_
>> freebsd-questions@freebsd.org mailing list
>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**questions
>> To unsubscribe, send any mail to "freebsd-questions-**
>> unsubscr...@freebsd.org "
>>
>
> __**_
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-**questions
> To unsubscribe, send any mail to "freebsd-questions-**
> unsubscr...@freebsd.org "
>



-- 
regards,
matt
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Da Rock]

On 02/11/12 01:34, Henry Olyer wrote:

So what do I change if I want to increase the shell's file limit?
I don't think you can. It's not a shell limit. It's a limit to the 
number of arguments the command itself will take. As said, the shell 
expands '*' to a list of files as the argument, and rm is limited to the 
number of arguments it will parse.

I use bash 4.

And by the way, for me, part of the normal installation of a new FBSD box
is to make certain changes.  For example, for "uniq -c" I use "%06" instead
of "%d" because this way I can sort the output.  Things like that.

I never learned a shell language.  I suppose no one is as dumb as someone
who choose's not to learn, so, what's the right one.  csh?, because I do a
lot of scientific work?, or should I be looking at another?
There's not really much difference in this factor for shell types; as 
for changes you'd have to hack the command's (say rm) code.


As mentioned, I'd use the find -delete combination.

On Wed, Feb 8, 2012 at 10:25 PM, andrew clarke  wrote:


On Tue 2012-02-07 23:17:16 UTC+, RW (rwmailli...@googlemail.com)
wrote:


On Tue, 07 Feb 2012 22:14:56 +
Matthew Seaman wrote:


ls -1 | xargs rm

but be aware that that wont work for filenames with spaces.

In addition, I don't believe it solves the OP's initial problem of the
argument list being too long!  You'd probably need to use the xargs -n
switch here.

The above will also try to 'rm' directories, which won't work.

Instead I would use 'find':

find . -type f -depth 1 -delete

This will also work with filenames with spaces.

Or the scenic route, using xargs, with one rm per file (slower):

find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f

(The "scenic route" is useful if you want to do something else with
the files instead of deleting them with rm.)

Regards
Andrew
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "
freebsd-questions-unsubscr...@freebsd.org"


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Henry Olyer]

So what do I change if I want to increase the shell's file limit?

I use bash 4.

And by the way, for me, part of the normal installation of a new FBSD box
is to make certain changes.  For example, for "uniq -c" I use "%06" instead
of "%d" because this way I can sort the output.  Things like that.

I never learned a shell language.  I suppose no one is as dumb as someone
who choose's not to learn, so, what's the right one.  csh?, because I do a
lot of scientific work?, or should I be looking at another?



On Wed, Feb 8, 2012 at 10:25 PM, andrew clarke  wrote:

> On Tue 2012-02-07 23:17:16 UTC+, RW (rwmailli...@googlemail.com)
> wrote:
>
> > On Tue, 07 Feb 2012 22:14:56 +
> > Matthew Seaman wrote:
> >
> > > ls -1 | xargs rm
> >
> > but be aware that that wont work for filenames with spaces.
>
> In addition, I don't believe it solves the OP's initial problem of the
> argument list being too long!  You'd probably need to use the xargs -n
> switch here.
>
> The above will also try to 'rm' directories, which won't work.
>
> Instead I would use 'find':
>
> find . -type f -depth 1 -delete
>
> This will also work with filenames with spaces.
>
> Or the scenic route, using xargs, with one rm per file (slower):
>
> find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f
>
> (The "scenic route" is useful if you want to do something else with
> the files instead of deleting them with rm.)
>
> Regards
> Andrew
> ___
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "
> freebsd-questions-unsubscr...@freebsd.org"
>
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[andrew clarke]

On Tue 2012-02-07 23:17:16 UTC+, RW (rwmailli...@googlemail.com) wrote:

> On Tue, 07 Feb 2012 22:14:56 +
> Matthew Seaman wrote:
> 
> > ls -1 | xargs rm
> 
> but be aware that that wont work for filenames with spaces.

In addition, I don't believe it solves the OP's initial problem of the
argument list being too long!  You'd probably need to use the xargs -n
switch here.

The above will also try to 'rm' directories, which won't work.

Instead I would use 'find':

find . -type f -depth 1 -delete

This will also work with filenames with spaces.

Or the scenic route, using xargs, with one rm per file (slower):

find . -type f -depth 1 -print0 | xargs -n1 -0 rm -f

(The "scenic route" is useful if you want to do something else with
the files instead of deleting them with rm.)

Regards
Andrew
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Robert Huff]

David Brodbeck writes:

>  > What helps me sometimes is wrapping it up:
>  >
>  > for i in *; do rm $i; done
>  
>  Won't that just expand the * and result in the same problem?  It
>  seems like you've just moved the problem from the rm statement to
>  the for statement.

If the problem is the command line to rm being too long, this
will work.
Yes, the '*" will get expanded to the list of files ... but
that will happen _within_ the running shell.  (Using the services of
glob(3) or something similar.)
The command line to "rm" will have a single file-name, and
should not be a problem.


Robert Huff

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[CyberLeo Kitsana]

On 02/08/2012 12:02 PM, David Brodbeck wrote:
> 2012/2/7 Ingo Hofmann :
>> What helps me sometimes is wrapping it up:
>>
>> for i in *; do rm $i; done
> 
> Won't that just expand the * and result in the same problem?  It seems
> like you've just moved the problem from the rm statement to the for
> statement.

This error arises during exec(3) when the length of the program
arguments exceeds a certain size. Since 'for' is a shell builtin, there
is no such practical limitation thereupon.

See the ERRORS section in execve(2), specifically [E2BIG], for more details.

-- 
Fuzzy love,
-CyberLeo
Technical Administrator
CyberLeo.Net Webhosting
http://www.CyberLeo.Net


Furry Peace! - http://.fur.com/peace/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[David Brodbeck]

2012/2/7 Ingo Hofmann :
> What helps me sometimes is wrapping it up:
>
> for i in *; do rm $i; done

Won't that just expand the * and result in the same problem?  It seems
like you've just moved the problem from the rm statement to the for
statement.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Modulok]

On 2/7/12, Коньков Евгений  wrote:
> # rm *
> /bin/rm: Argument list too long.
>
>
> in this directory about 25000 files,
> but actually there is only one argument to rm it is '*' sign.
>
> Why rm get list of all files in directore instead of deleting one by one?

If you're removing everything, can you just remove the directory and re-create
it? e.g:

cd ..
rm -rf foo
mkdir foo

-Modulok-
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[RW]

On Tue, 07 Feb 2012 22:14:56 +
Matthew Seaman wrote:

> ls -1 | xargs rm

but be aware that that wont work for filenames with spaces.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Randal L. Schwartz]

> "Matthew" == Matthew Seaman  writes:

Matthew> As you have discovered, it is very easy to overload the argument list.
Matthew> There are many ways around this, but one of the best ones is to use
Matthew> xargs(1). eg:

Matthew>% ls -1 | xargs rm

No need for the -1 there.  Whenever ls is not going to the terminal, it
defaults to classic -1 behavior.

Compare:

% ls

with

% ls | cat

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
 http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[CK]

Êîíüêîâ Åâãåíèé wrote:

# rm *
/bin/rm: Argument list too long.


in this directory about 25000 files,
but actually there is only one argument to rm it is '*' sign.

Why rm get list of all files in directore instead of deleting one by one?


Short answer: this is not Windows.

Long answer: shell does wildcard expands, therefore rm gets about 25000 
arguments of expanded '*'. Try


find ./ -type f -delete

Thanks,
CK
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Ingo Hofmann]

What helps me sometimes is wrapping it up:

for i in *; do rm $i; done

Best,
Ingo

P.S.:
Helps also with whitespaces in the filename where 'rm *' fails too.

On 07.02.2012, at 14:10 , Rares Aioanei wrote:

> On 02/07/2012 11:59 PM, Коньков Евгений wrote:
>> # rm *
>> /bin/rm: Argument list too long.
>> 
>> 
>> in this directory about 25000 files,
>> but actually there is only one argument to rm it is '*' sign.
>> 
>> Why rm get list of all files in directore instead of deleting one by one?
>> 
>> ___
>> freebsd-questions@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
>> 
> AFAIR, the shell expands * to every match , so from this point of view it is 
> equivalent to the 25000 files. Try 'rm -f [a-d]*' , then [e-h]* or similar 
> until you get them all.
> 
> Best,
> 
> -- 
> Rares Aioanei
> 
> ___
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Andre Goree]

2012/2/7 Коньков Евгений 

> # rm *
> /bin/rm: Argument list too long.
>
>
> in this directory about 25000 files,
> but actually there is only one argument to rm it is '*' sign.
>
> Why rm get list of all files in directore instead of deleting one by one?
>
> ___
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "
> freebsd-questions-unsubscr...@freebsd.org"
>


The 'rm' command lists each file in the directory prior to removing it.
 For very large directories, the proper (and faster in other cases) way is
to user 'find', such as below:

#  find . -delete

or, if you only want to delete files (leaving the directories):

# find . -type f -delete
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Matthew Seaman]

On 07/02/2012 21:59, Коньков Евгений wrote:
> # rm *
> /bin/rm: Argument list too long.
> 
> 
> in this directory about 25000 files,
> but actually there is only one argument to rm it is '*' sign.
> 
> Why rm get list of all files in directore instead of deleting one by one?

It's the shell that expands wild cards, and it will attempt to fork and
exec rm(1) with an arg list of all matching files. rm(1) itself has no
concept of wildcards -- it expects a list of filenames.

As you have discovered, it is very easy to overload the argument list.
There are many ways around this, but one of the best ones is to use
xargs(1). eg:

% ls -1 | xargs rm

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: 'rm' Can not delete files

[Rares Aioanei]

On 02/07/2012 11:59 PM, Коньков Евгений wrote:

# rm *
/bin/rm: Argument list too long.


in this directory about 25000 files,
but actually there is only one argument to rm it is '*' sign.

Why rm get list of all files in directore instead of deleting one by one?

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

AFAIR, the shell expands * to every match , so from this point of view 
it is equivalent to the 25000 files. Try 'rm -f [a-d]*' , then [e-h]* or 
similar until you get them all.


Best,

--
Rares Aioanei

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: 'rm' Can not delete files

[Randal L. Schwartz]

> "Коньков" == Коньков Евгений  writes:

Коньков> # rm *
Коньков> /bin/rm: Argument list too long.


Коньков> in this directory about 25000 files,
Коньков> but actually there is only one argument to rm it is '*' sign.

Коньков> Why rm get list of all files in directore instead of deleting
Коньков> one by one?

It's the way Unix works.  Individual commands typically know nothing of
'*' or '[abc]'.  The shell "globs" the command, expanding it to all the
matching files, and then hands that list to the command as individual
arguments.

However, sometimes that expansion exceeds the maximum argument size, as
you discovered.

If you have perl, try this:

  perl -e 'unlink glob("*")'

It uses Perl's internal glob() function, which can handle an unlimited
number of files.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
 http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"