Re: [gentoo-user] Any way around Argument list too long?
Top-posting IMPORTANT NOTE ON TOP Do NOT create the file I mention below unless you WANT to risk deleting ALL your files. On Tuesday 19 July 2011 15:58:44 Florian Philipp wrote: The double dash will prevent mv from interpreting weird file names like -h as parameters. Just about every standard GNU tool supports this. Or worse, a file called -rf . If anyone wants to try what happens when rm * finds that file, please do so ONLY on a machine you want to wipe anyway... -- Joost
Re: [gentoo-user] Any way around Argument list too long?
Am 18.07.2011 10:48, schrieb Neil Bothwick:
On Sun, 17 Jul 2011 18:40:44 -0700, Grant wrote:
Alright, find is tricky. Is this the right spot for -delete?
/usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday'
+\%Y\%m\%d`*.jpg - delete
Yes, but if you don't want irreversible mistakes, move the files instead.
find /home/user -type f -name blah -exec mv -t ~/.Trashcan {} +
In all these commands it would always be a good idea to deactivate
parameter parsing just in front of the place where the file names are
inserted.
find ... -print0 | xargs -0 mv -t ~/.Trashcan --
or
find ... -exec mv -t ~/.Trashcan -- {} +
The double dash will prevent mv from interpreting weird file names like
-h as parameters. Just about every standard GNU tool supports this.
Regards,
Florian Philipp
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-user] Any way around Argument list too long?
On Tue, 19 Jul 2011 15:58:44 +0200, Florian Philipp wrote: In all these commands it would always be a good idea to deactivate parameter parsing just in front of the place where the file names are inserted. Very good point! -- Neil Bothwick This is the day for firm decisions! Or is it? signature.asc Description: PGP signature
Re: [gentoo-user] Any way around Argument list too long?
Alright, find is tricky. Is this the right spot for -delete?
/usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday'
+\%Y\%m\%d`*.jpg - delete
Yes, but if you don't want irreversible mistakes, move the files instead.
find /home/user -type f -name blah -exec mv -t ~/.Trashcan {} +
In all these commands it would always be a good idea to deactivate
parameter parsing just in front of the place where the file names are
inserted.
find ... -print0 | xargs -0 mv -t ~/.Trashcan --
or
find ... -exec mv -t ~/.Trashcan -- {} +
The double dash will prevent mv from interpreting weird file names like
-h as parameters. Just about every standard GNU tool supports this.
Does that apply to a command like this:
/usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday'
+\%Y\%m\%d`*.jpg -delete
Maybe it should be changed to this:
/usr/bin/find /home/user -type f -name -- *-`/bin/date -d 'yesterday'
+\%Y\%m\%d`*.jpg -delete
- Grant
Re: [gentoo-user] Any way around Argument list too long?
On Tue, 19 Jul 2011 11:24:55 -0700, Grant wrote about Re: [gentoo-user] Any way around Argument list too long?: [snip] The double dash will prevent mv from interpreting weird file names like -h as parameters. Just about every standard GNU tool supports this. Does that apply to a command like this: /usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg -delete No. The find command is far too complicated for such a simple mechanism to make sense. Stopping the parser on such a character sequence has the potential for disaster. Maybe it should be changed to this: /usr/bin/find /home/user -type f -name -- *-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg -delete If you wish to delete JPEG files that are a day or more old, you might be better off using the ctime field in the directory entry: find /home/user -daystart -ctime 0 -type f -name \*.jpg -delete To see which files would be deleted, run the above command with -delete replaced by -print. Note also that the back-quote notation for command sub-shells has been deprecated since about 1993. See: http://compute.cnr.berkeley.edu/cgi-bin/man-cgi?ksh+1 and scroll down to the bottom of page 11. You are better off using the more modern $(command ...) notation. -- Regards, Dave [RLU #314465] *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* dwn...@ntlworld.com (David W Noon) *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* signature.asc Description: PGP signature
Re: [gentoo-user] Any way around Argument list too long?
On Sun, 17 Jul 2011 18:40:44 -0700, Grant wrote:
Alright, find is tricky. Is this the right spot for -delete?
/usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday'
+\%Y\%m\%d`*.jpg - delete
Yes, but if you don't want irreversible mistakes, move the files instead.
find /home/user -type f -name blah -exec mv -t ~/.Trashcan {} +
--
Neil Bothwick
The facts, although interesting, are usually irrelevant.
signature.asc
Description: PGP signature
Re: [gentoo-user] Any way around Argument list too long?
Alright, find is tricky. Is this the right spot for -delete?
/usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday'
+\%Y\%m\%d`*.jpg - delete
Yes, but if you don't want irreversible mistakes, move the files instead.
find /home/user -type f -name blah -exec mv -t ~/.Trashcan {} +
Thanks Neil and everyone.
- Grant
[gentoo-user] Any way around Argument list too long?
My crontab deletes all files of a certain type in a certain folder with yesterday's date in the filename. It usually executes but sometimes fails with: /bin/rm: Argument list too long What would you do about this? - Grant
Re: [gentoo-user] Any way around Argument list too long?
Hi, Grant. On Sun, Jul 17, 2011 at 12:32:42PM -0700, Grant wrote: My crontab deletes all files of a certain type in a certain folder with yesterday's date in the filename. It usually executes but sometimes fails with: /bin/rm: Argument list too long What would you do about this? Use xargs - in place of /bin/rm lots of files .. Use Lots_of_file_names | xargs rm . xargs then calls rm several times with batches of filenames each time. xargs is a standard Unix command. - Grant -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] Any way around Argument list too long?
it could be slightly less efficient comparing to plain `rm', but worked around
your problem:
find your-dir -ctime -1 -exec rm {} \;
basically `find' has a lot options filtering result set. these include
time/date, file name (regexp), file type and so on. consult man for details
victor
Grant wrote, at 07/17/2011 11:32 PM:
My crontab deletes all files of a certain type in a certain folder
with yesterday's date in the filename. It usually executes but
sometimes fails with:
/bin/rm: Argument list too long
What would you do about this?
- Grant
Re: [gentoo-user] Any way around Argument list too long?
On Sun, 17 Jul 2011 12:32:42 -0700, Grant wrote about [gentoo-user] Any way around Argument list too long?: My crontab deletes all files of a certain type in a certain folder with yesterday's date in the filename. It usually executes but sometimes fails with: /bin/rm: Argument list too long What would you do about this? Use find with the -delete option. -- Regards, Dave [RLU #314465] *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* dwn...@ntlworld.com (David W Noon) *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* signature.asc Description: PGP signature
RE: [gentoo-user] Any way around Argument list too long?
-original message-
Subject: Re: [gentoo-user] Any way around Argument list too long?
From: Alan Mackenzie a...@muc.de
Date: 2011-07-18 02:42
Hi, Grant.
On Sun, Jul 17, 2011 at 12:32:42PM -0700, Grant wrote:
My crontab deletes all files of a certain type in a certain folder
with yesterday's date in the filename. It usually executes but
sometimes fails with:
/bin/rm: Argument list too long
What would you do about this?
Use xargs - in place of
/bin/rm lots of files ..
Use
Lots_of_file_names | xargs rm
. xargs then calls rm several times with batches of filenames each time.
xargs is a standard Unix command.
You'll want to be extra careful with special characters like (space), single
quote, and double quote.
Better use find . -exec rm {} +
See: http://en.m.wikipedia.org/wiki/Xargs
Rgds,
--
FdS Pandu E Poluan
~ IT Optimizer ~
Sent from Nokia E72-1
Re: [gentoo-user] Any way around Argument list too long?
Are you using wildcards in the arguments to rm ? Rather use find | xargs or find -exec which are designed to deal with exactly this circumstance. On 17 Jul 2011 9:32 PM, Grant emailgr...@gmail.com wrote: My crontab deletes all files of a certain type in a certain folder with yesterday's date in the filename. It usually executes but sometimes fails with: /bin/rm: Argument list too long What would you do about this? - Grant
Re: [gentoo-user] Any way around Argument list too long?
On Sun, Jul 17, 2011 at 3:54 PM, Pandu Poluan pa...@poluan.info wrote:
-original message-
Subject: Re: [gentoo-user] Any way around Argument list too long?
From: Alan Mackenzie a...@muc.de
Date: 2011-07-18 02:42
Hi, Grant.
On Sun, Jul 17, 2011 at 12:32:42PM -0700, Grant wrote:
My crontab deletes all files of a certain type in a certain folder
with yesterday's date in the filename. It usually executes but
sometimes fails with:
/bin/rm: Argument list too long
What would you do about this?
Use xargs - in place of
/bin/rm lots of files ..
Use
Lots_of_file_names | xargs rm
. xargs then calls rm several times with batches of filenames each time.
xargs is a standard Unix command.
You'll want to be extra careful with special characters like (space), single
quote, and double quote.
Better use find . -exec rm {} +
This is why I use 'find' with the -print0 parameter.
find (path) (filespec) -print0|xargs -0 command
--
:wq
Re: [gentoo-user] Any way around Argument list too long?
way around Argument list too long?: My crontab deletes all files of a certain type in a certain folder with yesterday's date in the filename. It usually executes but sometimes fails with: /bin/rm: Argument list too long What would you do about this? Use find with the -delete option. I'm getting the same thing from find: $ /usr/bin/find /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg /usr/bin/find: Argument list too long $ /usr/bin/find -delete /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg /usr/bin/find: Argument list too long $ /usr/bin/find /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg|xargs rm /usr/bin/find: Argument list too long rm: missing operand Try `rm --help' for more information. - Grant
Re: [gentoo-user] Any way around Argument list too long?
On Sun, Jul 17, 2011 at 7:23 PM, Grant emailgr...@gmail.com wrote: I'm getting the same thing from find: $ /usr/bin/find /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg /usr/bin/find: Argument list too long You're using find wrong; the first argument needs to be the root path it starts searching from. To find files matching a particular name or pattern, try using -name or -iname. DATE=$(/bin/date -d 'yesterday' +\%Y\%m\%d) find /home/user -name '*-$DATE' -- :wq
Re: [gentoo-user] Any way around Argument list too long?
On Sunday 17 July 2011 16:23:54 Grant did opine thusly: way around Argument list too long?: My crontab deletes all files of a certain type in a certain folder with yesterday's date in the filename. It usually executes but sometimes fails with: /bin/rm: Argument list too long What would you do about this? Use find with the -delete option. I'm getting the same thing from find: $ /usr/bin/find /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg /usr/bin/find: Argument list too long $ /usr/bin/find -delete /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg /usr/bin/find: Argument list too long $ /usr/bin/find /home/user/*-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg|xargs rm /usr/bin/find: Argument list too long rm: missing operand Try `rm --help' for more information. You are doing it wrong. Each command has something between ``, so bash is expanding that entire list and just before feeding the whole lot to find, realizes that the list is longer than 65,536 characters. It's bash that is returning that error, not find. You're mistake is trying to marrow down *where* find should look instead of *what* it should look for. You something like this: find /home/user -type f -name `/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg` See the difference? That will not produce a gigantic command line, it will produce a rather short one and find will check each file it finds one by one and see if it's name matches the supplied pattern. Word of warning: DO NOT blindly run -delete on this, first check the total output and make sure it only has what you want to delete. As with all things concerning rm or file deletion, the burden rests on you to make completely sure you delete only what you want to delete. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Any way around Argument list too long?
You are doing it wrong. Each command has something between ``, so bash is expanding that entire list and just before feeding the whole lot to find, realizes that the list is longer than 65,536 characters. It's bash that is returning that error, not find. You're mistake is trying to marrow down *where* find should look instead of *what* it should look for. You something like this: find /home/user -type f -name `/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg` See the difference? That will not produce a gigantic command line, it will produce a rather short one and find will check each file it finds one by one and see if it's name matches the supplied pattern. Word of warning: DO NOT blindly run -delete on this, first check the total output and make sure it only has what you want to delete. As with all things concerning rm or file deletion, the burden rests on you to make completely sure you delete only what you want to delete. I ran this and the output was voluminous but looked good: /usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg So I ran it again, adding -delete right before -type. After a lot of processing I got a line of output like this for each file: /usr/bin/find: `/home/user/1-2011071612345.jpg': No such file or directory Unfortunately the command actually deleted the entire /home/user folder. Can anyone tell me what went wrong? Maybe '/home/user' was at the very top of the long list that scrolled up the screen when I ran the find command without -delete? - Grant
Re: [gentoo-user] Any way around Argument list too long?
On Sunday, July 17 at 17:47 (-0700), Grant said: ran this and the output was voluminous but looked good: /usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg So I ran it again, adding -delete right before -type. After a lot of That was a mistake. processing I got a line of output like this for each file: /usr/bin/find: `/home/user/1-2011071612345.jpg': No such file or directory Unfortunately the command actually deleted the entire /home/user folder. Can anyone tell me what went wrong? Maybe '/home/user' was at the very top of the long list that scrolled up the screen when I ran the find command without -delete? Well this is an unfortunate way to learn how find works. A better way would be: $ man find Basically find works of a chain of selection criteria. It crawls all the files/dirs and when one item in the chain is true for the criteria, it checks for the other. For example $ find /path -type f -name blah -print Crawls /path, for each file/dir it checks if it is a regular file (-type f), if that is true, it checks if it's name is blah, if that is true, it prints the name (blah). Therefore, $ find /path -delete -type f -name Crawls path, then checks -delete.. but wait, -delete evaluates to true if removal succeeded (find(1)), so it deletes the file, then checks to see if it is a regular file, then if that is true then it checks the name... but all that doesn't matter because your files are deleted. You should never put -delete at the beginning of a chain and, arguably, you shouldn't use -delete at all. It even says in the man page: Warnings: Don't forget that the find command line is evaluated as an expression, so putting -delete first will make find try to delete everything below the starting points you specified. When testing a find command line that you later intend to use with -delete, you should explicitly specify -depth in order to avoid later surprises. Because -delete implies -depth, you cannot usefully use -prune and -delete together.
Re: [gentoo-user] Any way around Argument list too long?
ran this and the output was voluminous but looked good: /usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg So I ran it again, adding -delete right before -type. After a lot of That was a mistake. processing I got a line of output like this for each file: /usr/bin/find: `/home/user/1-2011071612345.jpg': No such file or directory Unfortunately the command actually deleted the entire /home/user folder. Can anyone tell me what went wrong? Maybe '/home/user' was at the very top of the long list that scrolled up the screen when I ran the find command without -delete? Well this is an unfortunate way to learn how find works. A better way would be: $ man find Basically find works of a chain of selection criteria. It crawls all the files/dirs and when one item in the chain is true for the criteria, it checks for the other. For example $ find /path -type f -name blah -print Crawls /path, for each file/dir it checks if it is a regular file (-type f), if that is true, it checks if it's name is blah, if that is true, it prints the name (blah). Therefore, $ find /path -delete -type f -name Crawls path, then checks -delete.. but wait, -delete evaluates to true if removal succeeded (find(1)), so it deletes the file, then checks to see if it is a regular file, then if that is true then it checks the name... but all that doesn't matter because your files are deleted. You should never put -delete at the beginning of a chain and, arguably, you shouldn't use -delete at all. It even says in the man page: Warnings: Don't forget that the find command line is evaluated as an expression, so putting -delete first will make find try to delete everything below the starting points you specified. When testing a find command line that you later intend to use with -delete, you should explicitly specify -depth in order to avoid later surprises. Because -delete implies -depth, you cannot usefully use -prune and -delete together. Alright, find is tricky. Is this the right spot for -delete? /usr/bin/find /home/user -type f -name *-`/bin/date -d 'yesterday' +\%Y\%m\%d`*.jpg - delete - Grant
