Thanks Debra. I know there's a lot of info I dumped in my original email so maybe my question/message wasn't clear.
I'm still confused over this. I only started dabbling with the flush settings because I wasn't getting more than about 56% on the tape. I can't see how setting it back will change that. When I add up what flushed and what's not flushed, it appears as if it would all fit on the tape. Is there any way of testing this in a so called 'dry run'? Otherwise I'll be waiting weeks to see what a couple of tweaks here and there will actually do. On 21/10/14 08:28, Debra S Baddorf wrote: > Here’s a thought: > orig: >>> flush-threshold-dumped 100 >>> flush-threshold-scheduled 100 >>> taperflush 100 >>> autoflush yes > now: >>> flush-threshold-dumped 50 >>> flush-threshold-scheduled 100 >>> taperflush 0 >>> autoflush yes > You now allow amanda to start writing to tape when only 50% of the data is > ready. > (flush-threshold-dumped). Previously, 100% had to be ready — and THAT > allows > the best fit of DLE’s onto tape. Ie: > - pick the biggest DLE that will fit. Write it to tape. > - repeat. > > Now, the biggest one may not be done yet. But you’ve already started > writing all the > small pieces onto the tape, so maybe when you reach the Big Guy, there is > no space > for it. > The “Greedy Algorithm” (above: pick biggest. repeat) works best when > all the > parts are available for it to choose. > > Try setting flush-threshold-dumped back to 100. It won’t write as SOON — > cuz it waits > till 100% of a tape is available, but it might FILL the tape better. > > I think. > > Deb Baddorf > Fermilab > > On Oct 20, 2014, at 3:44 PM, Tom Robinson <[email protected]> wrote: > >> Anyone care to comment? >> >> On 20/10/14 10:49, Tom Robinson wrote: >>> Hi, >>> >>> I'm not sure why I'm not getting such good tape usage any more and wonder >>> if someone can help me. >>> >>> Until recently I was getting quite good tape usage on my 'weekly' config: >>> >>> USAGE BY TAPE: >>> Label Time Size % DLEs Parts >>> weekly01 3:10 1749362651K 117.9 16 16 >>> weekly02 3:09 1667194493K 112.4 21 21 >>> weekly03 3:08 1714523420K 115.5 16 16 >>> weekly04 3:04 1664570982K 112.2 21 21 >>> weekly05 3:11 1698357067K 114.5 17 17 >>> weekly06 3:07 1686467027K 113.7 21 21 >>> weekly07 3:03 1708584546K 115.1 17 17 >>> weekly08 3:11 1657764181K 111.7 21 21 >>> weekly09 3:03 1725209913K 116.3 17 17 >>> weekly10 3:12 1643311109K 110.7 21 21 >>> weekly01 3:06 1694157008K 114.2 17 17 >>> >>> For that last entry, the mail report looked like this: >>> >>> These dumps were to tape weekly01. >>> Not using all tapes because 1 tapes filled; runtapes=1 does not allow >>> additional tapes. >>> There are 198378440K of dumps left in the holding disk. >>> They will be flushed on the next run. >>> >>> Which was fairly typical and to be expected since the tune of flush >>> settings was: >>> >>> flush-threshold-dumped 100 >>> flush-threshold-scheduled 100 >>> taperflush 100 >>> autoflush yes >>> >>> Now, without expectation, the dumps started to look like this: >>> >>> weekly02 3:21 1289271529K 86.9 10 10 >>> weekly03 3:17 854362421K 57.6 11 11 >>> weekly04 3:20 839198404K 56.6 11 11 >>> weekly05 9:40 637259676K 42.9 5 5 >>> weekly06 10:54 806737591K 54.4 15 15 >>> weekly09 1:12 35523072K 2.4 1 1 >>> weekly09 3:21 841844504K 56.7 11 11 >>> weekly01 3:16 842557835K 56.8 19 19 >>> >>> About the time it started looking different, I introduced a second config >>> for 'archive' but I can't >>> see why that would affect my 'weekly' run. >>> >>> I had a couple of bad runs and had to flush them manually and I'm not sure >>> what happened with tapes >>> weekly07 and weekly08 (they appear to be missing) and weekly09 is dumped to >>> twice in succession. >>> This looks very weird. >>> >>> $ amadmin weekly find | grep weekly07 >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >>> weekly07 >>> 1 >>> 1/-1 PARTIAL PARTIAL >>> $ amadmin weekly find | grep weekly08 >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >>> weekly08 >>> 1 >>> 1/-1 PARTIAL PARTIAL >>> $ amadmin weekly find | grep weekly09 >>> 2014-09-21 00:00:00 monza / 0 >>> weekly09 >>> 9 >>> 1/1 OK >>> 2014-09-21 00:00:00 monza /data/backup/amanda/vtapes/daily/slot1 0 >>> weekly09 >>> 10 >>> 1/1 OK >>> 2014-09-21 00:00:00 monza /data/backup/amanda/vtapes/daily/slot2 0 >>> weekly09 >>> 11 >>> 1/-1 OK PARTIAL >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >>> weekly09 >>> 1 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot5 0 >>> weekly09 >>> 2 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot6 0 >>> weekly09 >>> 3 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot7 0 >>> weekly09 >>> 4 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot8 0 >>> weekly09 >>> 5 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /export 0 >>> weekly09 >>> 6 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /export/home 0 >>> weekly09 >>> 7 >>> 1/1 OK >>> 2014-09-14 00:00:00 monza /export/home/tom 0 >>> weekly09 >>> 8 >>> 1/1 OK >>> >>> >>> More recently (about three weesk ago) I upgraded the OS. I don't think it >>> has anything to do with >>> this but mention it for completeness. >>> >>> To get as much on tape as possible I was originally using: >>> >>> flush-threshold-dumped 100 >>> flush-threshold-scheduled 100 >>> taperflush 100 >>> autoflush yes >>> >>> But now, in an effort to tune better tape usage, I've dabbled with the >>> settings. My full amanda.conf >>> is below. I include some configs (include statements) but have only shown >>> robots.conf and >>> tapetypes.conf as the dumptypes.conf and networks.conf are pretty much >>> stock standard and haven't >>> been modified. >>> >>> Kind regards, >>> Tom >>> >>> >>> #amanda.conf >>> org "somedomain.com weekly" >>> mailto "[email protected]" >>> dumpuser "amanda" >>> inparallel 4 >>> dumporder "sssS" >>> taperalgo first >>> displayunit "k" >>> netusage 8000 Kbps >>> dumpcycle 8 weeks >>> runspercycle 8 >>> tapecycle 10 tapes >>> bumpsize 20 Mb >>> bumppercent 20 >>> bumpdays 1 >>> bumpmult 4 >>> etimeout 3000 >>> dtimeout 1800 >>> ctimeout 30 >>> device_output_buffer_size 81920k >>> usetimestamps yes >>> flush-threshold-dumped 50 >>> flush-threshold-scheduled 100 >>> taperflush 0 >>> autoflush yes >>> runtapes 1 >>> includefile "/etc/opt/csw/amanda/robot.conf" >>> maxdumpsize -1 >>> tapetype ULT3580-TD5 >>> labelstr "^weekly[0-9][0-9]*$" >>> amrecover_changer "changer" >>> holdingdisk hd1 { >>> comment "main holding disk" >>> directory "/data/spool/amanda/hold/monza" >>> use -100 Mb >>> chunksize 1Gb >>> } >>> infofile "/etc/opt/csw/amanda/weekly/curinfo" >>> logdir "/etc/opt/csw/amanda/weekly" >>> indexdir "/etc/opt/csw/amanda/weekly/index" >>> includefile "/etc/opt/csw/amanda/dumptypes.conf" >>> includefile "/etc/opt/csw/amanda/networks.conf" >>> includefile "/etc/opt/csw/amanda/tapetypes.conf" >>> >>> #robot.conf >>> define changer robot { >>> tpchanger "chg-robot:/dev/scsi/changer/c1t5000E11156304003d1" >>> property "tape-device" "0=tape:/dev/rmt/0bn" >>> #property "eject-before-unload" "yes" >>> property "use-slots" "1-23" >>> device-property "BLOCK_SIZE" "512k" >>> device-property "READ_BLOCK_SIZE" "512k" >>> device-property "FSF_AFTER_FILEMARK" "false" >>> device-property "LEOM" "TRUE" >>> } >>> tapedev "robot" >>> >>> # tapetypes.conf >>> define tapetype global { >>> part_size 3G >>> part_cache_type none >>> } >>> >>> define tapetype ULT3580-TD5 { >>> comment "Created by amtapetype; compression enabled" >>> length 1483868160 kbytes >>> filemark 868 kbytes >>> speed 85837 kps >>> blocksize 512 kbytes >>> } >>> >>> >>> >> >
signature.asc
Description: OpenPGP digital signature
