Dump + GZIP
Hi all, Can I safely pump a filesystem dump through gzip during the dumping process?, or di I need to create the dump first then gzip it after? Does zipping the dumps cause any headaches at restore time? (I currently dump 5 servers worth of data to a raid 5 array, and am about 20% away from running out of disk space). Does gzipping a file give a decent compression ratio? -Grant ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dump + GZIP
On Thursday 16 August 2007, Grant Peel wrote: Can I safely pump a filesystem dump through gzip during the dumping process?, or di I need to create the dump first then gzip it after? I do it all the time: dump -f - ... | gzip date_filesystem.dump.gz or with bzip2: dump -f - ... | bzip2 date_filesystem.dump.bz2 Does zipping the dumps cause any headaches at restore time? Nope: bzcat date_filesystem.dump.bz2 | restore ... -f - (I currently dump 5 servers worth of data to a raid 5 array, and am about 20% away from running out of disk space). Does gzipping a file give a decent compression ratio? Depends on what you're compressing, but generally yes. bzip2 generally compresses better but takes a lot more time, CPU and memory at compression time. JN ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dump + GZIP
On 8/16/07, John Nielsen [EMAIL PROTECTED] wrote: On Thursday 16 August 2007, Grant Peel wrote: Can I safely pump a filesystem dump through gzip during the dumping process?, or di I need to create the dump first then gzip it after? I do it all the time: dump -f - ... | gzip date_filesystem.dump.gz or with bzip2: dump -f - ... | bzip2 date_filesystem.dump.bz2 Does zipping the dumps cause any headaches at restore time? Nope: bzcat date_filesystem.dump.bz2 | restore ... -f - (I currently dump 5 servers worth of data to a raid 5 array, and am about 20% away from running out of disk space). Does gzipping a file give a decent compression ratio? Depends on what you're compressing, but generally yes. bzip2 generally compresses better but takes a lot more time, CPU and memory at compression time. Try give LZMA a shot. The last time I checked, it was a lot faster then bzip2 at decompression and made smaller files too. For speed however, gzip would be the best choice. Federico ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dump + GZIP
On Thu, Aug 16, 2007 at 09:10:12AM -0400, John Nielsen wrote:
On Thursday 16 August 2007, Grant Peel wrote:
Can I safely pump a filesystem dump through gzip during the dumping
process?, or di I need to create the dump first then gzip it after?
I do it all the time: dump -f - ... | gzip date_filesystem.dump.gz
or with bzip2: dump -f - ... | bzip2 date_filesystem.dump.bz2
Unless you're dumping an unmounted filesystem, add '-L' and '-h 0'. The
first is for dumping a snapshot of a live filesystem, and the second one
is to honor the nodump flag for level 0 dumps.
Does zipping the dumps cause any headaches at restore time?
Nope: bzcat date_filesystem.dump.bz2 | restore ... -f -
(I currently dump 5 servers worth of data to a raid 5 array, and am about
20% away from running out of disk space).
Does gzipping a file give a decent compression ratio?
Depends on what you're compressing, but generally yes. bzip2 generally
compresses better but takes a lot more time, CPU and memory at compression
time.
Some time ago I did some tests. Compression with gzip saved 50-60% on
dumps of /, /usr and /var. However, the savings for my /home partition
which contains a lot of digital camera pictures in JPEG format was only
11%.
While bzip2 did 8-10% better (except on /home), it takes a _lot_
longer. So at first I decided to stick with gzip.
Later I decided to skip compression altogether. If the backup medium
becomes corrupted, you might still be able to restore the dump for a
large part. Corruption in a compressed file makes the rest of the file
unreadable.
If you don't compress the dumps, it's easy to split them in multiple DVD
sized parts. The size of a dump is given in kiB, and it must be a
multiple of the block size, which defaults to 10 kiB. A DVD is 4.7 GB =
(4.7e9/1024//10)*10 = 4589840 kiB. The following command creates
a dump of /home in multiple DVD-sized chunks;
dump -0 -B 4589840 -C 8 -P 'cat - home-vol${DUMP_VOLUME}.dump' -h 0 -L /home
You can burn these chunks to DVDs with growisofs, e.g:
growisofs -dvd-compat -Z /dev/dvd=home-vol0.dump
growisofs -dvd-compat -Z /dev/dvd=home-vol1.dump
...
In this case you do need temporary storage for the dump files. You could
conveivably modify the argument of -P to burn directly with growisofs.
One caution for backups though. Use only programs that are available on
a install CD or in /rescue. Your restore process should not depend on a
port that you need to install first!
Roland
--
R.F.Smith http://www.xs4all.nl/~rsmith/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
pgpuQ7r99LyUq.pgp
Description: PGP signature
