Re: Fwd: Help: tar & find
On Thu, Oct 23, 2003 at 08:25:20AM -0600, Steve D wrote: > On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote: > > I am having trouble combining the tar and find command. I want to > > tar and > > delete all .bak,.Bak,.BAK files. > > > > I am using the following command but keep receiving errors. > [...] > > The script is as follows > > = > > #! /bin/bash > > set +x > > TAR_DIR=/home/tarbackups; > > FILES_DIR=/home/common; > > tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ > > `find $FILES_DIR -xdev -type f -iname "*.bak"`; > > == > [...] > > Here is some error output returned: > > > > tar: jobs/ROOF: Cannot stat: No such file or directory > > tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or > > directory > > tar: LEWIS: Cannot stat: No such file or directory > > tar: CRES.bak: Cannot stat: No such file or directory > [...] > > --- --- --- > > Matthew Seaman <[EMAIL PROTECTED]> replied: > > The problem is that you have file/directory names like 'ROOF LAYOUTS' > > which contain spaces and possibly other filenames containing > > characters with syntactic significance to the shell. > > > > Try: > > > > find $FILES_DIR -xdev -type f -iname "*.bak -print0 | \ > > xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date > > +%F`.tar.gz > > > > --- --- --- > > Would the following approach also work? (Have sed surround each item > returned by the find command with single quotes?) > --- > #! /bin/bash > set +x > TAR_DIR=/home/tarbackups; > FILES_DIR=/home/common; > tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ > `find $FILES_DIR -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/"`; > --- > > or the backticks in the last line replaced with the newer alternative "$()": > > "$( find $FILES_DIR -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/" )" ; > > > Do the characters \ * $ in sed's argument need to be quoted further, > to protect them from interpretation by the shell? The "find" portion > of the command works correctly, as written above, on my FreeBSD machine > using /bin/sh or /usr/local/bin/bash, but I don't know why those > characters in sed's argument don't need to be further escaped. You've apparently got double quotes inside a double quoted string. That doesn't work. Trying to enclose the output of find in quote marks will sort of work, but it's generally found to be flaky. Especially when the filenames you're dealing with also contain quotation marks of various types or return characters. This is exactly why the '-print0' primitive for find(1) was invented: it puts out a list of file names separated by ascii NULL characters, which is one of the two ascii characters you can't get in a filename. (The other is '/' -- the directory separator). The other great advantage of xargs(1) is that it knows how many command line arguments it can supply to it's dependent command, and it will chop up a long argument list into managable chunks. Unfortunately the particular tar(1) command you're using can't be restarted in that way so it doesn't help here. You'ld only run into this problem if you were trying to backup many thousands of files, in which case your quoting the output of find trick would suffer in exactly the same way. There are many ways to solve that problem, should you encounter it. But I'll leave you to discover those for yourself if you need to. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way PGP: http://www.infracaninophile.co.uk/pgpkey Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK pgp0.pgp Description: PGP signature
Re: Fwd: Help: tar & find
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday 23 October 2003 15:34, Matthew Seaman wrote: > On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote: [skipped] > > The problem is that you have file/directory names like 'ROOF LAYOUTS' > which contain spaces and possibly other filenames containing > characters with syntactic significance to the shell. > > Try: > > find $FILES_DIR -xdev -type f -iname "*.bak -print0 | \ > xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date > +%F`.tar.gz It'll have some _bad_ features: If ``find'' will found to many files (default number of arguments to be passed by xargs to the ``utility'' (the tar command) is 5000, the default size of the command line is MAX_ARG (2048 bytes) //from man xargs), then xargs will run the ``utility'' twice, or more times,... consider it will be a very small differenses in time, you'll get that the last tar invocation 'll replace your archievs with that time stamp. IMHO. WBR, CBuH. - -- CBuH. CG[CX] XVyGYjau [EMAIL PROTECTED], ICQ#70929413 GnuPG(PGP) public key is: http://ccclike.chat.ru/my_public_key.asc -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQE/mf0L5Cj3gqxcdCoRAtvPAJ98ygwLGD1Oxprvl+TlOEEP99R7ygCeL8tR L5loOfn5U3sJDVpRzVG4MvY= =mJGX -END PGP SIGNATURE- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fwd: Help: tar & find
Steve D <[EMAIL PROTECTED]> wrote: > Would the following approach also work? (Have sed surround each > item returned by the find command with single quotes?) > --- > #! /bin/bash > set +x > TAR_DIR=/home/tarbackups; > FILES_DIR=/home/common; > tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ > `find $FILES_DIR -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/"`; > > or the backticks in the last line replaced with the newer > alternative "$()": > > "$( find $FILES_DIR -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/" )" ; > --- Matthew Seaman <[EMAIL PROTECTED]> responded: > > You've apparently got double quotes inside a double quoted string. > That doesn't work. > > Trying to enclose the output of find in quote marks will sort of > work, but it's generally found to be flaky. Especially when the > filenames you're dealing with also contain quotation marks of various > types or return characters. This is exactly why the '-print0' > primitive for find(1) was invented: it puts out a list of file names > separated by ascii NULL characters, which is one of the two ascii > characters you can't get in a filename. (The other is '/' -- the > directory separator). > [...] --- Thank you Matthew for this information. Interestingly, the double quotes inside double quotes seems to work on my machine (using /usr/local/bin/bash), perhaps because the contents inside the "$( )" are processed in a subshell? Output of terminal session follows: ~/tmp/test> touch 'filename with spaces.bak' 'file two.bak' file3.txt file4.BAK ~/tmp/test> ls -l total 0 -rw-r--r-- 1 xscd xscd 0 Oct 23 11:45 file two.bak -rw-r--r-- 1 xscd xscd 0 Oct 23 11:45 file3.txt -rw-r--r-- 1 xscd xscd 0 Oct 23 11:45 file4.BAK -rw-r--r-- 1 xscd xscd 0 Oct 23 11:45 filename with spaces.bak ~/tmp/test> ~/tmp/test> echo "$(find . -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/")" './filename with spaces.bak' './file two.bak' './file4.BAK' ~/tmp/test> Now because of your response I am motivated to read and learn more about find's -print0 option and about xargs. Thank you. Steve D Portales, NM US -- Civilization is a process in search of humanity. -Eli Khamarov ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fwd: Help: tar & find
On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote: > I am having trouble combining the tar and find command. I want to > tar and > delete all .bak,.Bak,.BAK files. > > I am using the following command but keep receiving errors. [...] > The script is as follows > = > #! /bin/bash > set +x > TAR_DIR=/home/tarbackups; > FILES_DIR=/home/common; > tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ > `find $FILES_DIR -xdev -type f -iname "*.bak"`; > == [...] > Here is some error output returned: > > tar: jobs/ROOF: Cannot stat: No such file or directory > tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or > directory > tar: LEWIS: Cannot stat: No such file or directory > tar: CRES.bak: Cannot stat: No such file or directory [...] --- --- --- Matthew Seaman <[EMAIL PROTECTED]> replied: > The problem is that you have file/directory names like 'ROOF LAYOUTS' > which contain spaces and possibly other filenames containing > characters with syntactic significance to the shell. > > Try: > > find $FILES_DIR -xdev -type f -iname "*.bak -print0 | \ > xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date > +%F`.tar.gz > --- --- --- Would the following approach also work? (Have sed surround each item returned by the find command with single quotes?) --- #! /bin/bash set +x TAR_DIR=/home/tarbackups; FILES_DIR=/home/common; tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ `find $FILES_DIR -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/"`; --- or the backticks in the last line replaced with the newer alternative "$()": "$( find $FILES_DIR -xdev -type f -iname "*.bak" | sed "s/\(^.*$\)/'\1'/" )" ; Do the characters \ * $ in sed's argument need to be quoted further, to protect them from interpretation by the shell? The "find" portion of the command works correctly, as written above, on my FreeBSD machine using /bin/sh or /usr/local/bin/bash, but I don't know why those characters in sed's argument don't need to be further escaped. --- --- --- Steve D Portales, NM US ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fwd: Help: tar & find
On Thu, Oct 23, 2003 at 12:34:40PM +0100, Matthew Seaman wrote: > On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote: > > > > I am having trouble combining the tar and find command. I want to tar > > and > > delete all .bak,.Bak,.BAK files. > > > > I am using the following command but keep receiving errors. The tar > > command > > appears to be receiving a truncated file/path (some of the time, but not > > always), and trying to procecss the file as two files. > > > > Any help on this would be great. > > > > The script is as follows > > = > > #! /bin/bash > > set +x > > TAR_DIR=/home/tarbackups; > > FILES_DIR=/home/common; > > tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ > > `find $FILES_DIR -xdev -type f -iname "*.bak"`; > > == > > The -xdev is to not desend into an external smb-mounted drive to a > > subdirectory. > > > > I have also tried substituting single quotes for the double quotes > > around the > > *.bak > > > > Here is some error output returned: > > > > tar: jobs/ROOF: Cannot stat: No such file or directory > > tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or > > directory > > tar: LEWIS: Cannot stat: No such file or directory > > tar: CRES.bak: Cannot stat: No such file or directory > > tar: /home/technical/nfsapc30-my_documents/1999: Cannot stat: No such > > file or > > directory > > tar: jobs/ROOF: Cannot stat: No such file or directory > > tar: LAYOUTS/RESIDENTIAL/FRASER/THODE: Cannot stat: No such file or > > directory > > tar: RES.bak: Cannot stat: No such file or directory > > The problem is that you have file/directory names like 'ROOF LAYOUTS' > which contain spaces and possibly other filenames containing > characters with syntactic significance to the shell. > > Try: > > find $FILES_DIR -xdev -type f -iname "*.bak -print0 | \ > xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz Let's try that again, and fix the typo: find $FILES_DIR -xdev -type f -iname "*.bak" -print0 | \ xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.g Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way PGP: http://www.infracaninophile.co.uk/pgpkey Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK pgp0.pgp Description: PGP signature
Re: Fwd: Help: tar & find
On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote: > > I am having trouble combining the tar and find command. I want to tar > and > delete all .bak,.Bak,.BAK files. > > I am using the following command but keep receiving errors. The tar > command > appears to be receiving a truncated file/path (some of the time, but not > always), and trying to procecss the file as two files. > > Any help on this would be great. > > The script is as follows > = > #! /bin/bash > set +x > TAR_DIR=/home/tarbackups; > FILES_DIR=/home/common; > tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ > `find $FILES_DIR -xdev -type f -iname "*.bak"`; > == > The -xdev is to not desend into an external smb-mounted drive to a > subdirectory. > > I have also tried substituting single quotes for the double quotes > around the > *.bak > > Here is some error output returned: > > tar: jobs/ROOF: Cannot stat: No such file or directory > tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or > directory > tar: LEWIS: Cannot stat: No such file or directory > tar: CRES.bak: Cannot stat: No such file or directory > tar: /home/technical/nfsapc30-my_documents/1999: Cannot stat: No such > file or > directory > tar: jobs/ROOF: Cannot stat: No such file or directory > tar: LAYOUTS/RESIDENTIAL/FRASER/THODE: Cannot stat: No such file or > directory > tar: RES.bak: Cannot stat: No such file or directory The problem is that you have file/directory names like 'ROOF LAYOUTS' which contain spaces and possibly other filenames containing characters with syntactic significance to the shell. Try: find $FILES_DIR -xdev -type f -iname "*.bak -print0 | \ xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way PGP: http://www.infracaninophile.co.uk/pgpkey Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK pgp0.pgp Description: PGP signature
Fwd: Help: tar & find
I am having trouble combining the tar and find command. I want to tar and delete all .bak,.Bak,.BAK files. I am using the following command but keep receiving errors. The tar command appears to be receiving a truncated file/path (some of the time, but not always), and trying to procecss the file as two files. Any help on this would be great. The script is as follows = #! /bin/bash set +x TAR_DIR=/home/tarbackups; FILES_DIR=/home/common; tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\ `find $FILES_DIR -xdev -type f -iname "*.bak"`; == The -xdev is to not desend into an external smb-mounted drive to a subdirectory. I have also tried substituting single quotes for the double quotes around the *.bak Here is some error output returned: tar: jobs/ROOF: Cannot stat: No such file or directory tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or directory tar: LEWIS: Cannot stat: No such file or directory tar: CRES.bak: Cannot stat: No such file or directory tar: /home/technical/nfsapc30-my_documents/1999: Cannot stat: No such file or directory tar: jobs/ROOF: Cannot stat: No such file or directory tar: LAYOUTS/RESIDENTIAL/FRASER/THODE: Cannot stat: No such file or directory tar: RES.bak: Cannot stat: No such file or directory Thanks, -- Scott A. Gerhardt, P.Geo. Gerhardt Information Technologies ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"