Re: [gentoo-user] xargs and rm funkiness
On Wednesday 26 May 2010 06:42:08 Joerg Schilling wrote:
Patrick Holthaus patrick.holth...@uni-bielefeld.de wrote:
You might try:
find -name *.ext -print0 | xargs -0 rm
But this is non-standard.
In what way is this non-standard? That is, what standard is it contrary to?
TMTOWTDI (There's More Than One Way To Do It) applies just as strongly to *nix
in general as it does to Perl. When there are multiple ways to do something,
it's often either a user preference issue or the method should be decided
based upon the particular details of the desired result. -exec may be a POSIX
standard function, but that doesn't mean it must be used over other options or
you're breaking the standard.
UNIX introduced -exec {} + 1990 (when David Korn rewrote find(1)
and it is in the POSIX standared since some time.
-exec (which potentially has problems with race conditions - -execdir should
almost always be used instead) runs the command once for each file found.
xargs will call the command once for as many files as it can fit on the command
line. For some instances, like rm, that probably isn't significant. But if
you're calling a complex process with lots of files, the overhead of starting
the many extra processes may be significant.
--
You have attributed conditions to villainy that simply result from
stupidity. - Robert A. Heinlein
Re: [gentoo-user] xargs and rm funkiness
On Saturday 29 May 2010, Daniel D Jones wrote:
On Wednesday 26 May 2010 06:42:08 Joerg Schilling wrote:
Patrick Holthaus patrick.holth...@uni-bielefeld.de wrote:
You might try:
find -name *.ext -print0 | xargs -0 rm
But this is non-standard.
In what way is this non-standard? That is, what standard is it contrary
to?
SUS (aka POSIX), although some people are pushing to include -print0 | xargs
-0 into the standard. What Joerg meant is that the above construct will only
run when using GNU find and xargs. Of course, if you're running Linux, that is
probably the case already anyway.
TMTOWTDI (There's More Than One Way To Do It) applies just as strongly
to *nix in general as it does to Perl. When there are multiple ways to do
something, it's often either a user preference issue or the method should
be decided based upon the particular details of the desired result. -exec
may be a POSIX standard function, but that doesn't mean it must be used
over other options or you're breaking the standard.
UNIX introduced -exec {} + 1990 (when David Korn rewrote find(1)
and it is in the POSIX standared since some time.
-exec (which potentially has problems with race conditions - -execdir
should almost always be used instead) runs the command once for each file
found.
If you use -exec {} + as he wrote, this is not true.
xargs will call the command once for as many files as it can fit on
the command line.
And so does -exec {} +
For some instances, like rm, that probably isn't
significant. But if you're calling a complex process with lots of files,
the overhead of starting the many extra processes may be significant.
See above.
Re: [gentoo-user] xargs and rm funkiness
Etaoin Shrdlu shr...@unlimitedmail.org wrote: On Saturday 29 May 2010, Daniel D Jones wrote: On Wednesday 26 May 2010 06:42:08 Joerg Schilling wrote: Patrick Holthaus patrick.holth...@uni-bielefeld.de wrote: You might try: find -name *.ext -print0 | xargs -0 rm But this is non-standard. In what way is this non-standard? That is, what standard is it contrary to? SUS (aka POSIX), although some people are pushing to include -print0 | xargs -0 into the standard. What Joerg meant is that the above construct will only run when using GNU find and xargs. Of course, if you're running Linux, that is probably the case already anyway. And there is a big cheavat against this proposal as xargs -0 was introduced long after -exec + exsists and as introducing xargs -0 would force us to change _many_ other utilities too in order to come to a consistent overall behavior again. For this reason, there was even the proposal to instead remove xargs from the standard as -exec + does everything that is needed. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] xargs and rm funkiness
On Saturday 29 May 2010 17:05:34 Daniel D Jones wrote:
On Wednesday 26 May 2010 06:42:08 Joerg Schilling wrote:
Patrick Holthaus patrick.holth...@uni-bielefeld.de wrote:
You might try:
find -name *.ext -print0 | xargs -0 rm
But this is non-standard.
In what way is this non-standard? That is, what standard is it contrary
to? TMTOWTDI (There's More Than One Way To Do It) applies just as strongly
to *nix in general as it does to Perl. When there are multiple ways to do
something, it's often either a user preference issue or the method should
be decided based upon the particular details of the desired result. -exec
may be a POSIX standard function, but that doesn't mean it must be used
over other options or you're breaking the standard.
UNIX introduced -exec {} + 1990 (when David Korn rewrote find(1)
and it is in the POSIX standared since some time.
-exec (which potentially has problems with race conditions - -execdir
should almost always be used instead) runs the command once for each file
found. xargs will call the command once for as many files as it can fit on
the command line. For some instances, like rm, that probably isn't
significant. But if you're calling a complex process with lots of files,
the overhead of starting the many extra processes may be significant.
Perhaps you don't know Joerg yet. When dealing with the man, it's important to
know where he's coming from - and that is not how Linux does stuff
He invariably refers to POSIX when mentioning standards. He uses this standard
to ensure that his code will work on any *nix platform. This puts him at odds
with the Linux crowd sometimes - two very different viewpoints.
It's not -exec that causes one processto be launched per item found, it is
-exec \;
He referred to -exec + which has the same behaviour as you mention - use as
many filenames as will fit on the command line.
--
alan dot mckinnon at gmail dot com
Re: [gentoo-user] xargs and rm funkiness
On Saturday 29 May 2010 14:59:16 Alan McKinnon wrote: On Saturday 29 May 2010 17:05:34 Daniel D Jones wrote: ... -exec (which potentially has problems with race conditions - -execdir should almost always be used instead) runs the command once for each file found. xargs will call the command once for as many files as it can fit on the command line. For some instances, like rm, that probably isn't significant. But if you're calling a complex process with lots of files, the overhead of starting the many extra processes may be significant. Perhaps you don't know Joerg yet. When dealing with the man, it's important to know where he's coming from - and that is not how Linux does stuff He invariably refers to POSIX when mentioning standards. He uses this standard to ensure that his code will work on any *nix platform. This puts him at odds with the Linux crowd sometimes - two very different viewpoints. I wasn't coming from a Linux perspective. I'm a network engineer. At work, I touch SSH servers running SunOS, file servers running BSD (don't recall what flavor off the top of my head - I'm not in them that often), terminals running HPUX and run Linux at home. xargs is available on all of them. It's not -exec that causes one processto be launched per item found, it is -exec \; He referred to -exec + which has the same behaviour as you mention - use as many filenames as will fit on the command line. You're correct, of course. I missed that in the man pages. Mea culpa. (I'm a network engineer, not a sysadmin.) -- If everybody knows such-and-such, then it ain't so, by at least ten thousand to one. - Robert A. Heinlein
Re: [gentoo-user] xargs and rm funkiness
Neil Bothwick n...@digimed.co.uk wrote:
xargs can suck with anything but plain ASCII-without-spaces filenames.,
and it quite unnecessary here.
find -name *.ext -exe rm {} \;
or maybe even
find -name *.ext -exe rm {} +
Just avoid xargs as it is the source of the proplem.
find -name *.ext -exec some-command {} +
Is the preferred method since 1990.
Jörg
--
EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
j...@cs.tu-berlin.de(uni)
joerg.schill...@fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] xargs and rm funkiness
Patrick Holthaus patrick.holth...@uni-bielefeld.de wrote:
You might try:
find -name *.ext -print0 | xargs -0 rm
But this is non-standard.
UNIX introduced -exec {} + 1990 (when David Korn rewrote find(1)
and it is in the POSIX standared since some time.
Jörg
--
EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
j...@cs.tu-berlin.de(uni)
joerg.schill...@fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] xargs and rm funkiness
On Saturday 22 May 2010 08:25:55 Neil Bothwick wrote:
On Sat, 22 May 2010 12:35:10 +0100, David W Noon wrote:
find -name *.ext -exe rm {} +
Or simpler still:
find -name *.ext -delete
Neat - I hadn't noticed that option.
Anyone for find / -delete ?
If you use the -delete switch, just be careful. From the man page:
QUOTE
Warnings: Don't forget that the find command line is evaluated as an
expression, so putting -delete first will make find try to delete
everything below the starting points you specified.
/QUOTE
That means that the command
find -name *.tmp -delete
will delete all your temp files while
find -delete -name *.tmp
will delete everything below your current directory. If you're in the root
directory, it's equivalent to running your suggested command above. I just
found this out the hard way, although luckily I wasn't in the root directory
when I ran the command and so didn't trash my system. I did lose the changes
I'd made to the project I was working on but fortunately had a backup of the
original files.
--
If you give me six lines written by the most honest man, I will find something
in them to hang him. - Cardinal Richelieu
Re: [gentoo-user] xargs and rm funkiness
On Fri, 21 May 2010 21:49:49 -0400, Daniel D Jones wrote:
find -name *.ext | xargs -0 rm
I get the result:
rm: cannot remove `Long File Name One.ext\nLong File Name Two.ext\nLong
File Name Three.ext\n': File name too long.
xargs can suck with anything but plain ASCII-without-spaces filenames.,
and it quite unnecessary here.
find -name *.ext -exe rm {} \;
or maybe even
find -name *.ext -exe rm {} +
--
Neil Bothwick
He who asks a question is a fool for a minute,
He who doesn't ask is a fool for a lifetime.
signature.asc
Description: PGP signature
Re: [gentoo-user] xargs and rm funkiness
Hey! On Saturday 22 May 2010 03:49:49 Daniel D Jones wrote: Running the command: find -name *.ext | xargs -0 rm You might try: find -name *.ext -print0 | xargs -0 rm Cheers Patrick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] xargs and rm funkiness
On Sat, 22 May 2010 10:10:02 +0200, Neil Bothwick wrote about Re:
[gentoo-user] xargs and rm funkiness:
On Fri, 21 May 2010 21:49:49 -0400, Daniel D Jones wrote:
find -name *.ext | xargs -0 rm
I get the result:
rm: cannot remove `Long File Name One.ext\nLong File Name
Two.ext\nLong File Name Three.ext\n': File name too long.
xargs can suck with anything but plain ASCII-without-spaces filenames.,
and it quite unnecessary here.
find -name *.ext -exe rm {} \;
or maybe even
find -name *.ext -exe rm {} +
Or simpler still:
find -name *.ext -delete
--
Regards,
Dave [RLU #314465]
==
dwn...@ntlworld.com (David W Noon)
==
signature.asc
Description: PGP signature
Re: [gentoo-user] xargs and rm funkiness
On Sat, 22 May 2010 12:35:10 +0100, David W Noon wrote:
find -name *.ext -exe rm {} +
Or simpler still:
find -name *.ext -delete
Neat - I hadn't noticed that option.
Anyone for find / -delete ?
--
Neil Bothwick
Top Oxymorons Number 31: Small crowd
signature.asc
Description: PGP signature
Re: [gentoo-user] xargs and rm funkiness
On Friday 21 May 2010 22:11:49 cov...@ccs.covici.com wrote: Daniel D Jones ddjo...@riddlemaster.org wrote: Running the command: find -name *.ext | xargs -0 rm I get the result: rm: cannot remove `Long File Name One.ext\nLong File Name Two.ext\nLong File Name Three.ext\n': File name too long. (The actual list is much longer than this, of course, or I wouldn't be using xargs.) For some reason, the \n isn't being recognized as a separator but rather as a part of a single long file name. Don't think $IFS would affect a command like rm but it doesn't appear to be the issue: ddjo...@merlin ~ $ set | grep IFS IFS=$' \t\n' I don't see any other ser variable which looks like a likely candidate to cause the behavior. Anyone have a clue what's going on? Why do you have -0 -- this replaces the \n's with a null character -- is that what you want? Not exactly. It doesn't replace anything. It tells xargs to look for a null to separate fields and to ignore the normal field separation characters. This is required if you have spaces in the field name, otherwise xargs sees the spaces as a field separator. xargs was doing exactly what I told it to do. Unfortunately, I didn't read the man pages and was relying on some poorly written web pages which indicated that -0 told xargs to skip spaces but didn't mention that it also told it to ignore \n. The solution, as Patrick Holthaus pointed out, was to use the -print0 argument with find, which instructs find to use the null character as a field separator in its output. -- The road of excess leads to the palace of wisdom. - William Blake
Re: [gentoo-user] xargs and rm funkiness
On Saturday 2010-05-22 09:46, Neil Bothwick wrote:
On Fri, 21 May 2010 21:49:49 -0400, Daniel D Jones wrote:
find -name *.ext | xargs -0 rm
I get the result:
rm: cannot remove `Long File Name One.ext\nLong File Name Two.ext\nLong
File Name Three.ext\n': File name too long.
xargs can suck with anything but plain ASCII-without-spaces filenames.,
That's why you use NUL terminators. Simple as that.
Also find's -exe calls rm *for each* file when used with \;.
find -name *.ext -exe rm {} \;
or maybe even
find -name *.ext -exe rm {} +
Re: [gentoo-user] xargs and rm funkiness
On Sat, 22 May 2010 20:38:53 +0200 (CEST), Jan Engelhardt wrote: xargs can suck with anything but plain ASCII-without-spaces filenames., That's why you use NUL terminators. Simple as that. That only helps with spaces, what if filenames contain other reserved characters? Also find's -exe calls rm *for each* file when used with \;. Which is why I suggested +, but the -delete option may be even better. -- Neil Bothwick Echo Speak: Whale oil beef hooked signature.asc Description: PGP signature
Re: [gentoo-user] xargs and rm funkiness
Daniel D Jones ddjo...@riddlemaster.org wrote: Running the command: find -name *.ext | xargs -0 rm I get the result: rm: cannot remove `Long File Name One.ext\nLong File Name Two.ext\nLong File Name Three.ext\n': File name too long. (The actual list is much longer than this, of course, or I wouldn't be using xargs.) For some reason, the \n isn't being recognized as a separator but rather as a part of a single long file name. Don't think $IFS would affect a command like rm but it doesn't appear to be the issue: ddjo...@merlin ~ $ set | grep IFS IFS=$' \t\n' I don't see any other ser variable which looks like a likely candidate to cause the behavior. Anyone have a clue what's going on? Why do you have -0 -- this replaces the \n's with a null character -- is that what you want? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
