My first suggestion is that whatever you use for file copying, run it
in a verbose mode and collect the output. Then use ls or find to
produce an audit of the files that should be in the backup and compare
that with the output of the file copying. Compare the two hierarchies
on disk, too!
In fact, you might want to do that first, before concluding that your
copying mechanism is broken now: du reports in blocks, and (a) you
might have different block sizes on source and target systems; (b)
minimum allocation size might be different on source and target
systems. Just doing an "ls -alR | sort " and collecting the outputs
from the source and target hierarchies should give you files you can
simply diff to check for missing files in the target.
Note that the output of du will not tell you if some files are
truncated if truncation is less than a block...
Using tar is definitely possible, and I'm not aware of any limitation
on the number of inodes it will copy, but some versions of tar run
into limits on the path + filename length (256 characters, if my old
brain cells don't deceive me) and if you have symbolic links in your
source directory you'll need to specify copying the links rather than
the files they point to. As Kathryn Smith noted, using tar with an
absolute path is a very bad practice, so don't do that.
Rsync should also work reasonably well, and with the right options
(and reasonably synchronous system clocks) will probably be reasonably
speedy: you likely will want to specify checking modification time
(file changes) or change time (inode changes) and only checking
checksums if there's a discrepancy.
If you have files with unusual structures, such as sparse files,
piping the output of the "find" command, with appropriate arguments,
into cpio (e.g. find [arguments] | cpio -pdlm [destination directory
path]) should also work. Cpio has an advantage if you're using mixed-
endian systems in that it can swap bytes or words as needed.
Solaris 10 ufsdump will let you select directories to be backed up
within a filesystem, so if you're on a current Solaris release, that's
another possibility.
On 8-Jan-09, at 4:06 PM, Richard 'Doc' Kinne wrote:
Hi Folks:
I'm looking at backups - simple backups right now.
We have a strategy where an old computer is mounted with a large
external, removable hard drive. Directories - large directories -
that we have on our other production servers are mounted on this
small computer via NFS. A cron job then does a simple "cp" from the
NFS mounted production drive partitions to to the large, external,
removable hard drive.
I thought it was an elegant solution, myself, except for one small,
niggling detail.
It doesn't work.
The process doesn't copy all the files. Oh, we're not having a
problem with file locks, no. When you do a "du -sh <directory>"
comparison between the /scsi/web directory on the backup drive and
the production /scsi/web directory the differences measure in the
GB. For example my production /scsi partition has 62GB on it. The
most recently done backup has 42GB on it!
What our research found is that the cp command apparently has a
limit of copying 250,000 inodes. I have image directories on the
webserver that have 114,000 files so this is the limit I think I'm
running into.
While I'm looking at solutions like Bacula and Amanda, etc., I'm
wondering if RSYNCing the files may work. Or will I run into the
same limitation?
Any thoughts?
---
Richard 'Doc' Kinne, [KQR]
American Association of Variable Star Observers
<rkinne @ aavso.org>
_______________________________________________
bblisa mailing list
[email protected]
http://www.bblisa.org/mailman/listinfo/bblisa
_______________________________________________
bblisa mailing list
[email protected]
http://www.bblisa.org/mailman/listinfo/bblisa
