Lot of interesting points raised from the various replies - thanks. I think I'm going to give it a go with:
some faster disks (hopefully a striped holding disk) 1024k block size Software compression client fast, to reduce data sent Hardware compression on (on tape) I also need to check the SAS card interface and slot etc. David -------------------- David Simpson - Computing Support Officer IBME - Institute of Biomedical Engineering Old Road Campus Research Building Oxford, OX3 7DQ Tel: 01865 617697 ext: 17697 -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Jon LaBadie Sent: 25 November 2015 22:36 To: [email protected] Subject: Re: Throughput/write to tape On Wed, Nov 25, 2015 at 01:22:24AM -0500, Gene Heskett wrote: > On Tuesday 24 November 2015 21:41:44 Jon LaBadie wrote: > ... > > You may wish to check your data. Create a huge file on that 2TB > > holding disk. Make it 10-20% of the tape capacity, say 700GB-1TB. > > Create it by "cat'ing" representative data together. Then tape the > > file using a dd in a loop something like this (untested). > > > > SECONDS=0 > > while dd if=<above_file> of=<your_tape_device> bs=<1M or 2M> > > do > > echo $SECONDS > > done > > echo $SECONDS > > > > The loop will terminate when dd reaches the end of the tape. > > You can then count up the number of dd's run (plus a partial) and > > figure out how much data was written and how long it took. > > > > Set your tape drive to use 1 or 2MB blocks (match above) and > > actually, make several runs under these conditions: > > > > 1. hw compress off -- gives total capacity of tape and time to tape > > 2. hw compress on -- should be more dd's run to see effect of hw > > comp > > > > Use gzip to compress the file > > > > 3. hw compress off -- should give same tape capacity as 1, and you can > > calculate if hw or sw compress is more > > effective 4. hw compress on -- can hw compress shrink it more? Is the > > combo > > better than the best of 2. or 3. > > This scenario is much easier said than done, Jon. The ONLY reliable > way to disable hw compression once it has been enabled, is to read out > the 32k tape label 1st block of the tape, rewind the tape, turn hw > compression off, and without doing anything that will move the tape, > re-write that label block back to the tape. Only that sequence seems > to reset the hw compression flag in the tape header. > > Since I've been using v-tapes for around a decade now, they don't have > hw compress ability. The recovery, should it be needed, is 10 to 100x > faster from v-tapes as the hard drive is random access, whereas the > tape is not. I am not setup to do offsite, but so far, knock on wood, > smartctl has given me more than enough warning of an impending drive > failure. > With current tape formats and Linux kernel drivers, is anyone still seeing the behavior to which Gene refers? Namely, that once a tape has been used with hardware compression, it was very difficult for amanda to use it in compression off mode. The problem stemed from two factors. Old drives (particularly DAT) were in compress mode or not for both reading and writing. If the drive was in compression off, and then read a compressed tape, the mode was switched back to on, even for writing. The second factor was amanda's protective behavior. It never writes to any tape until it reads it to check the tape is valid to use. For formats like AIT and LTO, I think the compression mode only affects writes. For those formats, if compression does not shrink a tape block, the block is written uncompressed. Thus when reading a tape, each block has to be checked for compression status and that status does not affect writing. Of course, not having those drives of those formats, I could be totally wrong. jl -- Jon H. LaBadie [email protected] 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
