Re: Fwd: Help: tar find

2004-01-26 Thread Matthew Seaman
On Thu, Oct 23, 2003 at 08:25:20AM -0600, Steve D wrote:
 On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote:
  I am having trouble combining the tar and find command.  I want to
  tar and
  delete all .bak,.Bak,.BAK files.
 
  I am using the following command but keep receiving errors.
 [...]
  The script is as follows
  =
  #! /bin/bash
  set +x
  TAR_DIR=/home/tarbackups;
  FILES_DIR=/home/common;
  tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
`find $FILES_DIR -xdev -type f -iname *.bak`;
  ==
 [...]
  Here is some error output returned:
 
  tar: jobs/ROOF: Cannot stat: No such file or directory
  tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or
  directory
  tar: LEWIS: Cannot stat: No such file or directory
  tar: CRES.bak: Cannot stat: No such file or directory
 [...]
 
 --- --- ---
 
 Matthew Seaman [EMAIL PROTECTED] replied:
  The problem is that you have file/directory names like 'ROOF LAYOUTS'
  which contain spaces and possibly other filenames containing
  characters with syntactic significance to the shell.
 
  Try:
 
  find $FILES_DIR -xdev -type f -iname *.bak -print0 | \
  xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date
  +%F`.tar.gz
 
 
 --- --- ---
 
 Would the following approach also work? (Have sed surround each item
 returned by the find command with single quotes?)
 ---
 #! /bin/bash
 set +x
 TAR_DIR=/home/tarbackups;
 FILES_DIR=/home/common;
 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
 `find $FILES_DIR -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/`;
 ---
 
 or the backticks in the last line replaced with the newer alternative $():
 
 $( find $FILES_DIR -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/ ) ;
 
  
 Do the characters \ * $ in sed's argument need to be quoted further,
 to protect them from interpretation by the shell? The find portion
 of the command works correctly, as written above, on my FreeBSD machine
 using /bin/sh or /usr/local/bin/bash, but I don't know why those 
 characters in sed's argument don't need to be further escaped.

You've apparently got double quotes inside a double quoted string.
That doesn't work.

Trying to enclose the output of find in quote marks will sort of work,
but it's generally found to be flaky.  Especially when the filenames
you're dealing with also contain quotation marks of various types or
return characters.  This is exactly why the '-print0' primitive for
find(1) was invented: it puts out a list of file names separated by
ascii NULL characters, which is one of the two ascii characters you
can't get in a filename. (The other is '/' -- the directory
separator).

The other great advantage of xargs(1) is that it knows how many
command line arguments it can supply to it's dependent command, and it
will chop up a long argument list into managable chunks.
Unfortunately the particular tar(1) command you're using can't be
restarted in that way so it doesn't help here. You'ld only run into
this problem if you were trying to backup many thousands of files, in
which case your quoting the output of find trick would suffer in
exactly the same way.

There are many ways to solve that problem, should you encounter it.
But I'll leave you to discover those for yourself if you need to.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   26 The Paddocks
  Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey Marlow
Tel: +44 1628 476614  Bucks., SL7 1TH UK


pgp0.pgp
Description: PGP signature


Re: Fwd: Help: tar find

2003-10-24 Thread CBuH.
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On Thursday 23 October 2003 15:34, Matthew Seaman wrote:
 On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote:
[skipped]

 The problem is that you have file/directory names like 'ROOF LAYOUTS'
 which contain spaces and possibly other filenames containing
 characters with syntactic significance to the shell.

 Try:

 find $FILES_DIR -xdev -type f -iname *.bak -print0 | \
 xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date
 +%F`.tar.gz

It'll have some _bad_ features:
If ``find'' will found to many files (default number of arguments to be passed 
by xargs to the ``utility'' (the tar command) is 5000, the default size of 
the command line is MAX_ARG (2048 bytes) //from man xargs), then xargs will 
run the ``utility'' twice, or more times,... consider it will be a very small 
differenses in time, you'll get that the last tar invocation 'll replace your 
archievs with that time stamp.

IMHO.

WBR, CBuH.

- -- 

CBuH. CG[CX] XVyGYjau [EMAIL PROTECTED], ICQ#70929413
GnuPG(PGP) public key is: http://ccclike.chat.ru/my_public_key.asc


-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.3 (FreeBSD)

iD8DBQE/mf0L5Cj3gqxcdCoRAtvPAJ98ygwLGD1Oxprvl+TlOEEP99R7ygCeL8tR
L5loOfn5U3sJDVpRzVG4MvY=
=mJGX
-END PGP SIGNATURE-

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Fwd: Help: tar find

2003-10-23 Thread Matthew Seaman
On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote:
 
 I am having trouble combining the tar and find command.  I want to tar 
 and
 delete all .bak,.Bak,.BAK files.
 
 I am using the following command but keep receiving errors.  The tar 
 command
 appears to be receiving a truncated file/path (some of the time, but not
 always), and trying to procecss the file as two files.
 
 Any help on this would be great.
 
 The script is as follows
 =
 #! /bin/bash
 set +x
 TAR_DIR=/home/tarbackups;
 FILES_DIR=/home/common;
 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
   `find $FILES_DIR -xdev -type f -iname *.bak`;
 ==
 The -xdev is to not desend into an external smb-mounted drive to a
 subdirectory.
 
 I have also tried substituting single quotes for the double quotes 
 around the
 *.bak
 
 Here is some error output returned:
 
 tar: jobs/ROOF: Cannot stat: No such file or directory
 tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or 
 directory
 tar: LEWIS: Cannot stat: No such file or directory
 tar: CRES.bak: Cannot stat: No such file or directory
 tar: /home/technical/nfsapc30-my_documents/1999: Cannot stat: No such 
 file or
 directory
 tar: jobs/ROOF: Cannot stat: No such file or directory
 tar: LAYOUTS/RESIDENTIAL/FRASER/THODE: Cannot stat: No such file or 
 directory
 tar: RES.bak: Cannot stat: No such file or directory

The problem is that you have file/directory names like 'ROOF LAYOUTS'
which contain spaces and possibly other filenames containing
characters with syntactic significance to the shell.

Try:

find $FILES_DIR -xdev -type f -iname *.bak -print0 | \
xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   26 The Paddocks
  Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey Marlow
Tel: +44 1628 476614  Bucks., SL7 1TH UK


pgp0.pgp
Description: PGP signature


Re: Fwd: Help: tar find

2003-10-23 Thread Matthew Seaman
On Thu, Oct 23, 2003 at 12:34:40PM +0100, Matthew Seaman wrote:
 On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote:
  
  I am having trouble combining the tar and find command.  I want to tar 
  and
  delete all .bak,.Bak,.BAK files.
  
  I am using the following command but keep receiving errors.  The tar 
  command
  appears to be receiving a truncated file/path (some of the time, but not
  always), and trying to procecss the file as two files.
  
  Any help on this would be great.
  
  The script is as follows
  =
  #! /bin/bash
  set +x
  TAR_DIR=/home/tarbackups;
  FILES_DIR=/home/common;
  tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
`find $FILES_DIR -xdev -type f -iname *.bak`;
  ==
  The -xdev is to not desend into an external smb-mounted drive to a
  subdirectory.
  
  I have also tried substituting single quotes for the double quotes 
  around the
  *.bak
  
  Here is some error output returned:
  
  tar: jobs/ROOF: Cannot stat: No such file or directory
  tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or 
  directory
  tar: LEWIS: Cannot stat: No such file or directory
  tar: CRES.bak: Cannot stat: No such file or directory
  tar: /home/technical/nfsapc30-my_documents/1999: Cannot stat: No such 
  file or
  directory
  tar: jobs/ROOF: Cannot stat: No such file or directory
  tar: LAYOUTS/RESIDENTIAL/FRASER/THODE: Cannot stat: No such file or 
  directory
  tar: RES.bak: Cannot stat: No such file or directory
 
 The problem is that you have file/directory names like 'ROOF LAYOUTS'
 which contain spaces and possibly other filenames containing
 characters with syntactic significance to the shell.
 
 Try:
 
 find $FILES_DIR -xdev -type f -iname *.bak -print0 | \
 xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz

Let's try that again, and fix the typo:

 find $FILES_DIR -xdev -type f -iname *.bak -print0 | \
 xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.g

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   26 The Paddocks
  Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey Marlow
Tel: +44 1628 476614  Bucks., SL7 1TH UK


pgp0.pgp
Description: PGP signature


Re: Fwd: Help: tar find

2003-10-23 Thread Steve D
On Wed, Oct 22, 2003 at 10:43:50PM -0600, Scott Gerhardt wrote:
 I am having trouble combining the tar and find command.  I want to
 tar and
 delete all .bak,.Bak,.BAK files.

 I am using the following command but keep receiving errors.
[...]
 The script is as follows
 =
 #! /bin/bash
 set +x
 TAR_DIR=/home/tarbackups;
 FILES_DIR=/home/common;
 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
   `find $FILES_DIR -xdev -type f -iname *.bak`;
 ==
[...]
 Here is some error output returned:

 tar: jobs/ROOF: Cannot stat: No such file or directory
 tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or
 directory
 tar: LEWIS: Cannot stat: No such file or directory
 tar: CRES.bak: Cannot stat: No such file or directory
[...]

--- --- ---

Matthew Seaman [EMAIL PROTECTED] replied:
 The problem is that you have file/directory names like 'ROOF LAYOUTS'
 which contain spaces and possibly other filenames containing
 characters with syntactic significance to the shell.

 Try:

 find $FILES_DIR -xdev -type f -iname *.bak -print0 | \
 xargs -0 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date
 +%F`.tar.gz


--- --- ---

Would the following approach also work? (Have sed surround each item
returned by the find command with single quotes?)
---
#! /bin/bash
set +x
TAR_DIR=/home/tarbackups;
FILES_DIR=/home/common;
tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
`find $FILES_DIR -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/`;
---

or the backticks in the last line replaced with the newer alternative $():

$( find $FILES_DIR -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/ ) ;

 
Do the characters \ * $ in sed's argument need to be quoted further,
to protect them from interpretation by the shell? The find portion
of the command works correctly, as written above, on my FreeBSD machine
using /bin/sh or /usr/local/bin/bash, but I don't know why those 
characters in sed's argument don't need to be further escaped.

--- --- ---

Steve D
Portales, NM US

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Fwd: Help: tar find

2003-10-23 Thread Steve D
Steve D [EMAIL PROTECTED] wrote:

 Would the following approach also work? (Have sed surround each
 item returned by the find command with single quotes?)
 ---
 #! /bin/bash
 set +x
 TAR_DIR=/home/tarbackups;
 FILES_DIR=/home/common;
 tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
 `find $FILES_DIR -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/`;

 or the backticks in the last line replaced with the newer
 alternative $():

 $( find $FILES_DIR -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/ ) ;

---

Matthew Seaman [EMAIL PROTECTED] responded:

 You've apparently got double quotes inside a double quoted string.
 That doesn't work.

 Trying to enclose the output of find in quote marks will sort of
 work, but it's generally found to be flaky.  Especially when the
 filenames you're dealing with also contain quotation marks of various
 types or return characters.  This is exactly why the '-print0'
 primitive for find(1) was invented: it puts out a list of file names
 separated by ascii NULL characters, which is one of the two ascii
 characters you can't get in a filename. (The other is '/' -- the
 directory separator).

[...]

---

Thank you Matthew for this information. Interestingly, the double quotes
inside double quotes seems to work on my machine (using
/usr/local/bin/bash), perhaps because the contents inside the $( ) 
are processed in a subshell? Output of terminal session follows:

   ~/tmp/test touch 'filename with spaces.bak' 'file two.bak' file3.txt file4.BAK
   ~/tmp/test ls -l
total 0
-rw-r--r--  1 xscd  xscd  0 Oct 23 11:45 file two.bak
-rw-r--r--  1 xscd  xscd  0 Oct 23 11:45 file3.txt
-rw-r--r--  1 xscd  xscd  0 Oct 23 11:45 file4.BAK
-rw-r--r--  1 xscd  xscd  0 Oct 23 11:45 filename with spaces.bak
   ~/tmp/test
   ~/tmp/test echo $(find . -xdev -type f -iname *.bak | sed s/\(^.*$\)/'\1'/)
'./filename with spaces.bak'
'./file two.bak'
'./file4.BAK'
   ~/tmp/test

Now because of your response I am motivated to read and learn more
about find's -print0 option and about xargs. Thank you.

Steve D
Portales, NM US

-- 

Civilization is a process in search of humanity. -Eli Khamarov


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Fwd: Help: tar find

2003-10-22 Thread Scott Gerhardt
I am having trouble combining the tar and find command.  I want to tar 
and
delete all .bak,.Bak,.BAK files.

I am using the following command but keep receiving errors.  The tar 
command
appears to be receiving a truncated file/path (some of the time, but not
always), and trying to procecss the file as two files.

Any help on this would be great.

The script is as follows
=
#! /bin/bash
set +x
TAR_DIR=/home/tarbackups;
FILES_DIR=/home/common;
tar --remove-files -cvzpf $TAR_DIR/bak_files_`date +%F`.tar.gz\
  `find $FILES_DIR -xdev -type f -iname *.bak`;
==
The -xdev is to not desend into an external smb-mounted drive to a
subdirectory.
I have also tried substituting single quotes for the double quotes 
around the
*.bak

Here is some error output returned:

tar: jobs/ROOF: Cannot stat: No such file or directory
tar: LAYOUTS/RESIDENTIAL/FRASER/219: Cannot stat: No such file or 
directory
tar: LEWIS: Cannot stat: No such file or directory
tar: CRES.bak: Cannot stat: No such file or directory
tar: /home/technical/nfsapc30-my_documents/1999: Cannot stat: No such 
file or
directory
tar: jobs/ROOF: Cannot stat: No such file or directory
tar: LAYOUTS/RESIDENTIAL/FRASER/THODE: Cannot stat: No such file or 
directory
tar: RES.bak: Cannot stat: No such file or directory



Thanks,

--
Scott A. Gerhardt, P.Geo.
Gerhardt Information Technologies
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]