On Saturday 15 June 2002 07:01, Niall O Broin wrote: >I'm trying to decide whether or not to use software compression in > my amanda configuration. At the moment, I'm backing up to a DDS-3 > and I've told Amanda that the tape capacity is 15GB to allow for > hardware compression. Of course I don't really know what level of > compression the drive is able to achieve with my mix of data. > >At the moment this is not an issue because the most data ever gets > taped in one run is ~10GB but as I'm adding clients and disks > this will increase and I'm concerned about maximising my use of > the limited capacity of the DDS-3 drive. Obviously using software > compression helps with this because Amanda knows exactly how big > each compressed dump is and I can also tell her the truth about > the tape capacity.
Using software compression allows amanda to have a much better view of how much a tape can hold, which is advantage 1. Advantage 2 is that you can turn the compression on and off according to the entries in your disklist. My /usr/dlds directory is mostly tar.gz and .bz2's, and will expand about 20% if a compressor is used, so that one goes to tape uncompressed. Set each new one up using compression and see the mail report from amanda, and turn the compression off by switching that entries tape profile if it expands on that particular directory tree. Time to compress, or uncompress isn't normally that big a deal unless the tape server is a 66mhz P1 or some such slowpoke. >However my big concern is restoring. I know that even with current >processors gzipping several GB of data takes some time and the > same applies in reverse - or does it ? Does it take significantly > longer to extract one file from a gzipped tar file on a DDS-3 > than it does to extract one file from an uncompressed tar file or > can a reasonable CPU gunzip in a pipe as fast as the DDS-3 can > deliver data ? Doubtfull unless the server is a 1.5+ ghz. OTOH, extraction time is normally lless than compression time by a large measure. >And then of course there's the fact that I won't be restoring very > often anyway, so the extra backup capacity obtained may be worth > the price of slower restores. > >Anyway, my dilemma should be clear. I'd appreciate feedback from > anyone else who's considered these issues. What did you decide, > and why ? > >A second question that arises is the issue of existing tapes. I've > read that once a DDS-3 tape has been written in hardware > compressed mode, the tape is marked accordingly and will ever > after be written with compression, no matter what the drive is > told to do. I've also read that this mark can only be removed by > using a magnetic tape eraser. Is this correct ? Using a mag tape eraser just wrecks the tape as it destroys the hidden from you factory tape header. The tape cannot be mounted or recognized by the drive, ever. However I have worked out a method that seems to work on such a tape, and it looks something like this: Become root Load the tape run this script saved as fix-compression like this: ./fix-compression -------------------------------------------- #!/bin/bash mt -f /dev/nst0 rewind # but don't rewind it here as you save the amanda label dd if=/dev/nst0 of=label-this-tape bs=512 count=1 # then (optional) so you can see it cat label-this-tape # turn it off mt -f /dev/nst0 compression off mt -f /dev/nst0 datcompression off mt -f /dev/nst0 defcompression off # write very long string of zero's to the tape, using rewinding dd of=/dev/st0 if=/dev/zero bs=512 count=32728 # optional, make sure its rewound mt -f /dev/nst0 tell # put the label back using the rewinding device dd if=label-this-tape of=/dev/st0 count=1 bs=512 # optional, make sure its rewound mt -f /dev/nst0 tell echo this slot is done ------------------eof, script------------- To clarify, the tapes OWN header that contains much of this compressed or not data will remain in the compressed state, but the flags that determine the status of the data on the tape will be reset and the rest of the tape will not be written in the compressed mode. I rescued about 20 dds2 tapes by doing this in a Seagate 4586np drive, aka a CTL-96. The compression led will come on while the label is being read, but will then go back off. And I'm not going to claim it works with any drive, these @^%# dats seem to write their own laws... -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.0% setiathome rank, not too shabby for a hillbilly
