Hmm, I did check my tape driver settings. When I set the tape library up, I spent a long time getting the driver settings right (on OmniOS) and took copious notes on the settings. My queries reveal that I'm not using compression (which is what I wanted as the vtapes are already compressed). LTO5 native is 1.5T; compressed is 3T.
All my tapes are 'newish' (about one year old). The tape unit is the same age. For months I was consistently getting over 110% (highest 117%), then capacity dropped once to 86% and then consistently to about 56% (lowest 42%). Is there a block size issue (2x56=112)? All the weekly dumps are local so network shouldn't be an issue. The tape unit is using redundant SAS connectors. Maybe it's a mpath thing? Should I re-run amtapetype to see what it thinks the 'new' tape capacity is after upgrading the OS? On 22/10/14 10:47, Debra S Baddorf wrote: > Yeah, it sure does look like it ought to fit! > Could the tapes be dirty and not holding as much any more??? I have no > idea if that’s even possible. > But it kinds seems like your tapes are saying “I don’t want that much data”. > Compression issues? > > Your tapes were previously getting 117% capacity, and now are only doing > 86%. Is that the general summary? > > Unless somebody else can suggest some network (cable to tape drive?) or > system problems which might make > the tapes appear smaller than before? Compression issues? Read the > original message > at the bottom of this email for the original problem complaint. > > Deb > > > On Oct 21, 2014, at 6:20 PM, Tom Robinson <[email protected]> wrote: > >> Hi Debra, >> >> A brilliant motivational speech. Thanks. Well worth the read. In homage, I >> strongly suggest anyone >> who hasn't read it to go and do that now. Here it is again for those whose >> mouse wheel is >> dysfunctional: >> >> http://www.appleseeds.org/Big-Rocks_Covey.htm >> >> I will try your suggestions but want to make clear that the virtual tapes >> you see are the product of >> a daily run which is disk only. The weekly run puts all those daily dumps >> onto tape which then >> leaves the building. So I have both virtual and real tapes. The issues I'm >> having are in the weekly >> run (the dump to real tape of a set of virtual tapes). >> >> The tapes can be viewed as a bunch of big/little rocks. The total amount of >> data, however they are >> stacked, should still fit on a single LTO5 tape (amtapetype told me: length >> 1483868160 kbytes): >> >> $ pwd >> /data/backup/amanda/vtapes/daily >> >> $ du -shc * >> 512 data >> 1.0K info >> 119G slot1 >> 144G slot2 >> 115G slot3 >> 101G slot4 >> 80G slot5 >> 157G slot6 >> 189G slot7 >> 117G slot8 >> 1019G total >> >> Plus: >> >> 4.2G / >> 212M /export >> 212M /export/home >> 212M /export/home/tom >> >> >> So, it looks like I do still have some big rocks to put in first but on the >> surface of it, it looks >> like it should all fit in anyway (Did I sum that up wrongly? Looks less than >> my tape length.). >> >> Thanks, >> Tom >> >> (BTW, your last email is not so much a diatribe as a oratory or allocution). >> >> On 22/10/14 03:31, Debra S Baddorf wrote: >>> Since nobody else is chiming in, I’ll have another go. >>> I don’t think there IS a dry-run of the taping process, since so much >>> depends on the timing >>> of when a DLE is finished and ready to go to tape, and the physical >>> fitting it onto tape >>> (although, since you have a virtual tape, presumably that isn’t as subject >>> to variation as >>> a real tape might be). >>> >>> I wonder if your root (or boot or sys or whatever you call them) >>> partitions are now just slightly >>> bigger, after your operating system upgrade. That would affect the way >>> things fit into the tape. >>> One has to put the biggest things in first, then the next biggest that >>> will still fit, etc >>> to make the most of the tape size. (see >>> http://www.appleseeds.org/Big-Rocks_Covey.htm >>> for the life motivational analysis type speech that uses this principal too) >>> >>> Yet you, Tom, are telling amanda to finish all the small things first, >>> and then put them onto tape >>> as soon as they are done: >>> dumporder “sssS” >>> taperalgo first >>> I have mine set to finish the big dumps first, so I can put them on the >>> tape first >>> dumporder “BTBTBTBTBT" >>> >>> Then — I want amanda to wait until it has a whole tapeful before it starts >>> writing — just so that >>> all those “big pieces” are done and available to be chosen. >>> flush-threshold-dumped 100 >>> >>> And THEN — I tell amanda to use the principle in the above motivational >>> speech — >>> PUT THE BIG THINGS IN FIRST to be sure they fit (and that I don’t have a >>> 40% space left >>> at the end of the tape which still isn’t big enough for that Big DLE that >>> just now finished). >>> taperalgo largestfit # pick the biggest file that will fit in space >>> left >>> #"Greedy Algorithm" -- best polynomial time choice >>> # (err, I think it was maybe my suggestion that >>> caused the creation of this option, >>> # cuz of the Knapsack problem & the Greedy >>> Algorithm from comp sic >>> # classes. Which is the same as the >>> motivational speech above.) Put the >>> # big stuff in first! Then you can always fit >>> the little stuff in the remaining space. >>> >>> SO TRY THIS: >>> If your operating system DLE is now big enough that it doesn’t fit in that >>> last 40% of the tape — >>> then make sure it is ready earlier >>> dumporder “BBB” or “BTBT” etc >>> and that the taper waits till it has a whole tape worth >>> flush-threshold-dumped 100 >>> AND that it chooses the biggest bits first >>> taperalgo largestfit. >>> >>> Make those three changes and see if it helps. I bet your tapes will again >>> be mostly full, and only >>> the little bits will be left over to flush next time. >>> >>> Deb Baddorf >>> Fermilab >>> >>> (ps the caps aren’t shouting — they are meant to make skimming this long >>> winded diatribe easier!) >>> >>> >>> On Oct 20, 2014, at 6:51 PM, Tom Robinson <[email protected]> wrote: >>> >>>> Hi Debra, >>>> >>>> Thanks for you comments especially regarding 'no record'. I did already >>>> make that setting in my >>>> disklist file for all DLEs. eg: >>>> >>>> host /path { >>>> root-tar >>>> strategy noinc >>>> record no >>>> } >>>> >>>> I didn't check it though until you mentioned it, so thanks again. >>>> >>>> I did read the man page regarding the settings for autoflush to >>>> distinguish the no/yes/all >>>> semantics. I chose specifically 'yes' ('With yes, only dump [sic] matching >>>> the command line argument >>>> are flushed.'). >>>> >>>> Since I'm using 'yes' and not 'all' for autoflush, I don't think that has >>>> been interfering. >>>> >>>> When I ran the manual flush I did have to override the flush settings >>>> because amanda didn't want to >>>> flush to tape at all. Just sat there waiting for more data, I guess. I >>>> didn't record the command and >>>> it's no longer in my history. From memory, I think it was: >>>> >>>> $ amflush -o flush-threshold-dumped=0 -o flush-threshold-scheduled=0 -o >>>> taperflush=0 -o autoflush=no >>>> weekly >>>> >>>> So essentially I was trying to flush with 'defaults' restored. Would that >>>> mess with my scheduled runs? >>>> >>>> Anyone have some clues about 'dry running' to see what tuning I need to >>>> tune without actually doing it? >>>> >>>> Regards, >>>> Tom >>>> >>>> >>>> On 21/10/14 10:27, Debra S Baddorf wrote: >>>>> Not an actual answer, but two comments: >>>>> 1- you’ve added a new config “archive”. Make sure you set it “no >>>>> record” so that >>>>> when IT does a level 0 of some disk, your normal config doesn’t read >>>>> that as ITS >>>>> level 0. The “level 0 was done <date>” info is not specific to the >>>>> configuration, >>>>> but to the disk itself. For a “dump” type dump (as opposed to tar) it >>>>> is stored in >>>>> /etc/dumpdates, and any dump done gets written there. Amanda’s >>>>> configurations are “meta data” >>>>> that amanda knows about but that the disk itself doesn’t know about. So >>>>> your >>>>> archive config might be changing the dump level patterns of your other >>>>> config, >>>>> unless you set the archive config to “no record”. >>>>> I’m not sure if this is affecting your current setup, but you did just >>>>> add that new config. >>>>> >>>>> 2- I became aware about a year ago that “autoflush yes” is no longer >>>>> the only >>>>> opposite to “autoflush no”. There is also a new-ish “autoflush all”. >>>>> If you type “amdump MyConfig” the either “yes” or “all” >>>>> should flush >>>>> everything. But if you type “amdump MyConfig aParticularNodeName” >>>>> then >>>>> it will only flush DLE’s that match that node name, unless you set it >>>>> to >>>>> “autoflush all”. >>>>> You did mention that you had to do a few flushes lately. If you >>>>> really meant that >>>>> you had to allow some DLE’s to auto-flush, then the “all” vs “yes” >>>>> might make a >>>>> difference to you. >>>>> >>>>> Other people: how can he do a “dry run” here? >>>>> >>>>> Deb >>>>> >>>>> On Oct 20, 2014, at 6:05 PM, Tom Robinson <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks Debra. I know there's a lot of info I dumped in my original email >>>>>> so maybe my >>>>>> question/message wasn't clear. >>>>>> >>>>>> I'm still confused over this. I only started dabbling with the flush >>>>>> settings because I wasn't >>>>>> getting more than about 56% on the tape. I can't see how setting it back >>>>>> will change that. >>>>>> >>>>>> When I add up what flushed and what's not flushed, it appears as if it >>>>>> would all fit on the tape. >>>>>> >>>>>> Is there any way of testing this in a so called 'dry run'? Otherwise >>>>>> I'll be waiting weeks to see >>>>>> what a couple of tweaks here and there will actually do. >>>>>> >>>>>> On 21/10/14 08:28, Debra S Baddorf wrote: >>>>>>> Here’s a thought: >>>>>>> orig: >>>>>>>>> flush-threshold-dumped 100 >>>>>>>>> flush-threshold-scheduled 100 >>>>>>>>> taperflush 100 >>>>>>>>> autoflush yes >>>>>>> now: >>>>>>>>> flush-threshold-dumped 50 >>>>>>>>> flush-threshold-scheduled 100 >>>>>>>>> taperflush 0 >>>>>>>>> autoflush yes >>>>>>> You now allow amanda to start writing to tape when only 50% of the data >>>>>>> is ready. >>>>>>> (flush-threshold-dumped). Previously, 100% had to be ready — and >>>>>>> THAT allows >>>>>>> the best fit of DLE’s onto tape. Ie: >>>>>>> - pick the biggest DLE that will fit. Write it to tape. >>>>>>> - repeat. >>>>>>> >>>>>>> Now, the biggest one may not be done yet. But you’ve already started >>>>>>> writing all the >>>>>>> small pieces onto the tape, so maybe when you reach the Big Guy, >>>>>>> there is no space >>>>>>> for it. >>>>>>> The “Greedy Algorithm” (above: pick biggest. repeat) works best >>>>>>> when all the >>>>>>> parts are available for it to choose. >>>>>>> >>>>>>> Try setting flush-threshold-dumped back to 100. It won’t write as >>>>>>> SOON — cuz it waits >>>>>>> till 100% of a tape is available, but it might FILL the tape better. >>>>>>> >>>>>>> I think. >>>>>>> >>>>>>> Deb Baddorf >>>>>>> Fermilab >>>>>>> >>>>>>> On Oct 20, 2014, at 3:44 PM, Tom Robinson <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Anyone care to comment? >>>>>>>> >>>>>>>> On 20/10/14 10:49, Tom Robinson wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm not sure why I'm not getting such good tape usage any more and >>>>>>>>> wonder if someone can help me. >>>>>>>>> >>>>>>>>> Until recently I was getting quite good tape usage on my 'weekly' >>>>>>>>> config: >>>>>>>>> >>>>>>>>> USAGE BY TAPE: >>>>>>>>> Label Time Size % DLEs Parts >>>>>>>>> weekly01 3:10 1749362651K 117.9 16 16 >>>>>>>>> weekly02 3:09 1667194493K 112.4 21 21 >>>>>>>>> weekly03 3:08 1714523420K 115.5 16 16 >>>>>>>>> weekly04 3:04 1664570982K 112.2 21 21 >>>>>>>>> weekly05 3:11 1698357067K 114.5 17 17 >>>>>>>>> weekly06 3:07 1686467027K 113.7 21 21 >>>>>>>>> weekly07 3:03 1708584546K 115.1 17 17 >>>>>>>>> weekly08 3:11 1657764181K 111.7 21 21 >>>>>>>>> weekly09 3:03 1725209913K 116.3 17 17 >>>>>>>>> weekly10 3:12 1643311109K 110.7 21 21 >>>>>>>>> weekly01 3:06 1694157008K 114.2 17 17 >>>>>>>>> >>>>>>>>> For that last entry, the mail report looked like this: >>>>>>>>> >>>>>>>>> These dumps were to tape weekly01. >>>>>>>>> Not using all tapes because 1 tapes filled; runtapes=1 does not allow >>>>>>>>> additional tapes. >>>>>>>>> There are 198378440K of dumps left in the holding disk. >>>>>>>>> They will be flushed on the next run. >>>>>>>>> >>>>>>>>> Which was fairly typical and to be expected since the tune of flush >>>>>>>>> settings was: >>>>>>>>> >>>>>>>>> flush-threshold-dumped 100 >>>>>>>>> flush-threshold-scheduled 100 >>>>>>>>> taperflush 100 >>>>>>>>> autoflush yes >>>>>>>>> >>>>>>>>> Now, without expectation, the dumps started to look like this: >>>>>>>>> >>>>>>>>> weekly02 3:21 1289271529K 86.9 10 10 >>>>>>>>> weekly03 3:17 854362421K 57.6 11 11 >>>>>>>>> weekly04 3:20 839198404K 56.6 11 11 >>>>>>>>> weekly05 9:40 637259676K 42.9 5 5 >>>>>>>>> weekly06 10:54 806737591K 54.4 15 15 >>>>>>>>> weekly09 1:12 35523072K 2.4 1 1 >>>>>>>>> weekly09 3:21 841844504K 56.7 11 11 >>>>>>>>> weekly01 3:16 842557835K 56.8 19 19 >>>>>>>>> >>>>>>>>> About the time it started looking different, I introduced a second >>>>>>>>> config for 'archive' but I can't >>>>>>>>> see why that would affect my 'weekly' run. >>>>>>>>> >>>>>>>>> I had a couple of bad runs and had to flush them manually and I'm not >>>>>>>>> sure what happened with tapes >>>>>>>>> weekly07 and weekly08 (they appear to be missing) and weekly09 is >>>>>>>>> dumped to twice in succession. >>>>>>>>> This looks very weird. >>>>>>>>> >>>>>>>>> $ amadmin weekly find | grep weekly07 >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >>>>>>>>> weekly07 >>>>>>>>> 1 >>>>>>>>> 1/-1 PARTIAL PARTIAL >>>>>>>>> $ amadmin weekly find | grep weekly08 >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >>>>>>>>> weekly08 >>>>>>>>> 1 >>>>>>>>> 1/-1 PARTIAL PARTIAL >>>>>>>>> $ amadmin weekly find | grep weekly09 >>>>>>>>> 2014-09-21 00:00:00 monza / 0 >>>>>>>>> weekly09 >>>>>>>>> 9 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-21 00:00:00 monza /data/backup/amanda/vtapes/daily/slot1 0 >>>>>>>>> weekly09 >>>>>>>>> 10 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-21 00:00:00 monza /data/backup/amanda/vtapes/daily/slot2 0 >>>>>>>>> weekly09 >>>>>>>>> 11 >>>>>>>>> 1/-1 OK PARTIAL >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot4 0 >>>>>>>>> weekly09 >>>>>>>>> 1 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot5 0 >>>>>>>>> weekly09 >>>>>>>>> 2 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot6 0 >>>>>>>>> weekly09 >>>>>>>>> 3 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot7 0 >>>>>>>>> weekly09 >>>>>>>>> 4 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /data/backup/amanda/vtapes/daily/slot8 0 >>>>>>>>> weekly09 >>>>>>>>> 5 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /export 0 >>>>>>>>> weekly09 >>>>>>>>> 6 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /export/home 0 >>>>>>>>> weekly09 >>>>>>>>> 7 >>>>>>>>> 1/1 OK >>>>>>>>> 2014-09-14 00:00:00 monza /export/home/tom 0 >>>>>>>>> weekly09 >>>>>>>>> 8 >>>>>>>>> 1/1 OK >>>>>>>>> >>>>>>>>> >>>>>>>>> More recently (about three weesk ago) I upgraded the OS. I don't >>>>>>>>> think it has anything to do with >>>>>>>>> this but mention it for completeness. >>>>>>>>> >>>>>>>>> To get as much on tape as possible I was originally using: >>>>>>>>> >>>>>>>>> flush-threshold-dumped 100 >>>>>>>>> flush-threshold-scheduled 100 >>>>>>>>> taperflush 100 >>>>>>>>> autoflush yes >>>>>>>>> >>>>>>>>> But now, in an effort to tune better tape usage, I've dabbled with >>>>>>>>> the settings. My full amanda.conf >>>>>>>>> is below. I include some configs (include statements) but have only >>>>>>>>> shown robots.conf and >>>>>>>>> tapetypes.conf as the dumptypes.conf and networks.conf are pretty >>>>>>>>> much stock standard and haven't >>>>>>>>> been modified. >>>>>>>>> >>>>>>>>> Kind regards, >>>>>>>>> Tom >>>>>>>>> >>>>>>>>> >>>>>>>>> #amanda.conf >>>>>>>>> org "somedomain.com weekly" >>>>>>>>> mailto "[email protected]" >>>>>>>>> dumpuser "amanda" >>>>>>>>> inparallel 4 >>>>>>>>> dumporder "sssS" >>>>>>>>> taperalgo first >>>>>>>>> displayunit "k" >>>>>>>>> netusage 8000 Kbps >>>>>>>>> dumpcycle 8 weeks >>>>>>>>> runspercycle 8 >>>>>>>>> tapecycle 10 tapes >>>>>>>>> bumpsize 20 Mb >>>>>>>>> bumppercent 20 >>>>>>>>> bumpdays 1 >>>>>>>>> bumpmult 4 >>>>>>>>> etimeout 3000 >>>>>>>>> dtimeout 1800 >>>>>>>>> ctimeout 30 >>>>>>>>> device_output_buffer_size 81920k >>>>>>>>> usetimestamps yes >>>>>>>>> flush-threshold-dumped 50 >>>>>>>>> flush-threshold-scheduled 100 >>>>>>>>> taperflush 0 >>>>>>>>> autoflush yes >>>>>>>>> runtapes 1 >>>>>>>>> includefile "/etc/opt/csw/amanda/robot.conf" >>>>>>>>> maxdumpsize -1 >>>>>>>>> tapetype ULT3580-TD5 >>>>>>>>> labelstr "^weekly[0-9][0-9]*$" >>>>>>>>> amrecover_changer "changer" >>>>>>>>> holdingdisk hd1 { >>>>>>>>> comment "main holding disk" >>>>>>>>> directory "/data/spool/amanda/hold/monza" >>>>>>>>> use -100 Mb >>>>>>>>> chunksize 1Gb >>>>>>>>> } >>>>>>>>> infofile "/etc/opt/csw/amanda/weekly/curinfo" >>>>>>>>> logdir "/etc/opt/csw/amanda/weekly" >>>>>>>>> indexdir "/etc/opt/csw/amanda/weekly/index" >>>>>>>>> includefile "/etc/opt/csw/amanda/dumptypes.conf" >>>>>>>>> includefile "/etc/opt/csw/amanda/networks.conf" >>>>>>>>> includefile "/etc/opt/csw/amanda/tapetypes.conf" >>>>>>>>> >>>>>>>>> #robot.conf >>>>>>>>> define changer robot { >>>>>>>>> tpchanger "chg-robot:/dev/scsi/changer/c1t5000E11156304003d1" >>>>>>>>> property "tape-device" "0=tape:/dev/rmt/0bn" >>>>>>>>> #property "eject-before-unload" "yes" >>>>>>>>> property "use-slots" "1-23" >>>>>>>>> device-property "BLOCK_SIZE" "512k" >>>>>>>>> device-property "READ_BLOCK_SIZE" "512k" >>>>>>>>> device-property "FSF_AFTER_FILEMARK" "false" >>>>>>>>> device-property "LEOM" "TRUE" >>>>>>>>> } >>>>>>>>> tapedev "robot" >>>>>>>>> >>>>>>>>> # tapetypes.conf >>>>>>>>> define tapetype global { >>>>>>>>> part_size 3G >>>>>>>>> part_cache_type none >>>>>>>>> } >>>>>>>>> >>>>>>>>> define tapetype ULT3580-TD5 { >>>>>>>>> comment "Created by amtapetype; compression enabled" >>>>>>>>> length 1483868160 kbytes >>>>>>>>> filemark 868 kbytes >>>>>>>>> speed 85837 kps >>>>>>>>> blocksize 512 kbytes >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> >>
signature.asc
Description: OpenPGP digital signature
