Here’s a thought: orig: >> flush-threshold-dumped 100 >> flush-threshold-scheduled 100 >> taperflush 100 >> autoflush yes now: >> flush-threshold-dumped 50 >> flush-threshold-scheduled 100 >> taperflush 0 >> autoflush yes
You now allow amanda to start writing to tape when only 50% of the data is ready. (flush-threshold-dumped). Previously, 100% had to be ready — and THAT allows the best fit of DLE’s onto tape. Ie: - pick the biggest DLE that will fit. Write it to tape. - repeat. Now, the biggest one may not be done yet. But you’ve already started writing all the small pieces onto the tape, so maybe when you reach the Big Guy, there is no space for it. The “Greedy Algorithm” (above: pick biggest. repeat) works best when all the parts are available for it to choose. Try setting flush-threshold-dumped back to 100. It won’t write as SOON — cuz it waits till 100% of a tape is available, but it might FILL the tape better. I think. Deb Baddorf Fermilab On Oct 20, 2014, at 3:44 PM, Tom Robinson <[email protected]> wrote: > Anyone care to comment? > > On 20/10/14 10:49, Tom Robinson wrote: >> Hi, >> >> I'm not sure why I'm not getting such good tape usage any more and wonder if >> someone can help me. >> >> Until recently I was getting quite good tape usage on my 'weekly' config: >> >> USAGE BY TAPE: >> Label Time Size % DLEs Parts >> weekly01 3:10 1749362651K 117.9 16 16 >> weekly02 3:09 1667194493K 112.4 21 21 >> weekly03 3:08 1714523420K 115.5 16 16 >> weekly04 3:04 1664570982K 112.2 21 21 >> weekly05 3:11 1698357067K 114.5 17 17 >> weekly06 3:07 1686467027K 113.7 21 21 >> weekly07 3:03 1708584546K 115.1 17 17 >> weekly08 3:11 1657764181K 111.7 21 21 >> weekly09 3:03 1725209913K 116.3 17 17 >> weekly10 3:12 1643311109K 110.7 21 21 >> weekly01 3:06 1694157008K 114.2 17 17 >> >> For that last entry, the mail report looked like this: >> >> These dumps were to tape weekly01. >> Not using all tapes because 1 tapes filled; runtapes=1 does not allow >> additional tapes. >> There are 198378440K of dumps left in the holding disk. >> They will be flushed on the next run. >> >> Which was fairly typical and to be expected since the tune of flush settings >> was: >> >> flush-threshold-dumped 100 >> flush-threshold-scheduled 100 >> taperflush 100 >> autoflush yes >> >> Now, without expectation, the dumps started to look like this: >> >> weekly02 3:21 1289271529K 86.9 10 10 >> weekly03 3:17 854362421K 57.6 11 11 >> weekly04 3:20 839198404K 56.6 11 11 >> weekly05 9:40 637259676K 42.9 5 5 >> weekly06 10:54 806737591K 54.4 15 15 >> weekly09 1:12 35523072K 2.4 1 1 >> weekly09 3:21 841844504K 56.7 11 11 >> weekly01 3:16 842557835K 56.8 19 19 >> >> About the time it started looking different, I introduced a second config >> for 'archive' but I can't >> see why that would affect my 'weekly' run. >> >> I had a couple of bad runs and had to flush them manually and I'm not sure >> what happened with tapes >> weekly07 and weekly08 (they appear to be missing) and weekly09 is dumped to >> twice in succession. >> This looks very weird. >> >> $ amadmin weekly find | grep weekly07 >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >> weekly07 >> 1 >> 1/-1 PARTIAL PARTIAL >> $ amadmin weekly find | grep weekly08 >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >> weekly08 >> 1 >> 1/-1 PARTIAL PARTIAL >> $ amadmin weekly find | grep weekly09 >> 2014-09-21 00:00:00 monza / 0 >> weekly09 >> 9 >> 1/1 OK >> 2014-09-21 00:00:00 monza /data/backup/amanda/vtapes/daily/slot1 0 >> weekly09 >> 10 >> 1/1 OK >> 2014-09-21 00:00:00 monza /data/backup/amanda/vtapes/daily/slot2 0 >> weekly09 >> 11 >> 1/-1 OK PARTIAL >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >> weekly09 >> 1 >> 1/1 OK >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot5 0 >> weekly09 >> 2 >> 1/1 OK >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot6 0 >> weekly09 >> 3 >> 1/1 OK >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot7 0 >> weekly09 >> 4 >> 1/1 OK >> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot8 0 >> weekly09 >> 5 >> 1/1 OK >> 2014-09-14 00:00:00 monza /export 0 >> weekly09 >> 6 >> 1/1 OK >> 2014-09-14 00:00:00 monza /export/home 0 >> weekly09 >> 7 >> 1/1 OK >> 2014-09-14 00:00:00 monza /export/home/tom 0 >> weekly09 >> 8 >> 1/1 OK >> >> >> More recently (about three weesk ago) I upgraded the OS. I don't think it >> has anything to do with >> this but mention it for completeness. >> >> To get as much on tape as possible I was originally using: >> >> flush-threshold-dumped 100 >> flush-threshold-scheduled 100 >> taperflush 100 >> autoflush yes >> >> But now, in an effort to tune better tape usage, I've dabbled with the >> settings. My full amanda.conf >> is below. I include some configs (include statements) but have only shown >> robots.conf and >> tapetypes.conf as the dumptypes.conf and networks.conf are pretty much stock >> standard and haven't >> been modified. >> >> Kind regards, >> Tom >> >> >> #amanda.conf >> org "somedomain.com weekly" >> mailto "[email protected]" >> dumpuser "amanda" >> inparallel 4 >> dumporder "sssS" >> taperalgo first >> displayunit "k" >> netusage 8000 Kbps >> dumpcycle 8 weeks >> runspercycle 8 >> tapecycle 10 tapes >> bumpsize 20 Mb >> bumppercent 20 >> bumpdays 1 >> bumpmult 4 >> etimeout 3000 >> dtimeout 1800 >> ctimeout 30 >> device_output_buffer_size 81920k >> usetimestamps yes >> flush-threshold-dumped 50 >> flush-threshold-scheduled 100 >> taperflush 0 >> autoflush yes >> runtapes 1 >> includefile "/etc/opt/csw/amanda/robot.conf" >> maxdumpsize -1 >> tapetype ULT3580-TD5 >> labelstr "^weekly[0-9][0-9]*$" >> amrecover_changer "changer" >> holdingdisk hd1 { >> comment "main holding disk" >> directory "/data/spool/amanda/hold/monza" >> use -100 Mb >> chunksize 1Gb >> } >> infofile "/etc/opt/csw/amanda/weekly/curinfo" >> logdir "/etc/opt/csw/amanda/weekly" >> indexdir "/etc/opt/csw/amanda/weekly/index" >> includefile "/etc/opt/csw/amanda/dumptypes.conf" >> includefile "/etc/opt/csw/amanda/networks.conf" >> includefile "/etc/opt/csw/amanda/tapetypes.conf" >> >> #robot.conf >> define changer robot { >> tpchanger "chg-robot:/dev/scsi/changer/c1t5000E11156304003d1" >> property "tape-device" "0=tape:/dev/rmt/0bn" >> #property "eject-before-unload" "yes" >> property "use-slots" "1-23" >> device-property "BLOCK_SIZE" "512k" >> device-property "READ_BLOCK_SIZE" "512k" >> device-property "FSF_AFTER_FILEMARK" "false" >> device-property "LEOM" "TRUE" >> } >> tapedev "robot" >> >> # tapetypes.conf >> define tapetype global { >> part_size 3G >> part_cache_type none >> } >> >> define tapetype ULT3580-TD5 { >> comment "Created by amtapetype; compression enabled" >> length 1483868160 kbytes >> filemark 868 kbytes >> speed 85837 kps >> blocksize 512 kbytes >> } >> >> >> > >
