* David Simpson <[email protected]> [20151118 11:35]: > Hi Deb, Jon > > Hopefully this will cover both responses. Thanks again... > > > Holdingdisk is already configured as 2000GB that as much as I can do on that > server. > Chunksize I have already set to 500MB > > Advice on 1/4 of tape size for max DE noted. > > > >>Archive - with infinite tapes: Amanda will chug through all the tapes in > >>your changer, and then will wait until you put some more new tapes in > >>there > (note to self: no it won’t, it will fail cuz it can’t find tapes. How can > we tell it to WAIT ON THE OPERATOR to change tapes?) > > Interested in this. Will the dump fail? What will happen will it stop with a > DLE in the Holdingdisk waiting to be flushed? > > > > ---------------------- > > I ran the tape profiler and got these numbers seen in my definition (I also > added part size into the definition). I don't know why I didn't get closer to > 170mps or 180mps.... kernel module/driver? > > define tapetype "HP-Ultrium-LTO6" { > comment "Created by amtapetype; compression enabled" > length 2299845440 kbytes > speed 69251 kps > blocksize 32 kbytes > #use default filemark - don't specify > #filemark > #needed for spanning... set low to try to avoid wastage (in abscence of > LEOM) > part_size 100G > }
Retry by disabling tape compression with mt and use bigger blocksizes, like 2MiB or even more. Note that a part_size of 100G will lead to 25 chunks on a tape if amanda fills it to capacity. That's a lot of stop-and-go for the tape heads. OTOH, you use tapes more efficiently...your call. cheers, jf > > > ------------------------- > > >>It sounds like you are thinking of a single full backup (your off-site > >>archive) and then just backing up the changes thereafter. That > > -I'm planning on a full backup (off site archive). > > -After that a full backup and proper cycle, likely a weekly or fortnightly > dumpcycle (so Amanda running in a more typical fashion) - but to do this... > I'm going to make a call on what I will and won't backup to try to reduce the > volume (probably to 7-8 tapes worth max). As Deb importantly point outs. > > > >> Using your above parameters, that would mean about 20% of a full dump each > >> run (plus incrementals). > > An important point. Perhaps it would be prudent to change to this (?) to > reduce the nightly volume: > > dumpcycle 2 weeks > runspercycle 10 > tapecycle 8 > > > > I would want to avoid spilling into the daytime (7 or 8am) with the regular > backup if possible. > > > > regards > David > > > > -------------------- > David Simpson - Computing Support Officer > IBME - Institute of Biomedical Engineering > Old Road Campus Research Building > Oxford, OX3 7DQ > Tel: 01865 617697 ext: 17697 > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On > Behalf Of Jon LaBadie > Sent: 18 November 2015 06:01 > To: Amanda Users <[email protected]> > Subject: Re: archive and near-line usage > > On Tue, Nov 17, 2015 at 11:27:30PM +0000, Debra S Baddorf wrote: > > AMANDA USERS LIST: anybody else want to jump in here with better > > ideas?? > > I’m open to better suggestions ….. I’m probably making a real hash of > > this, and > > not necessarily making suggestions in the best order….. > > Deb > > I'll throw in a few thoughts. > > > > > > On Nov 11, 2015, at 2:55 PM, David Simpson <[email protected]> > > > wrote: > > > > > > Hi all, > > > > > > Thanks for the help so far. This has saved me a lot of time. > > > > > > I have a fair amount of data I'm looking to copy (potentially ~30TB) > > > - that is data from multiple file-servers to multiple tapes and without > > > LEOM. > > > Am I correct in saying this is the implication(?) - Amanda will get > > > so far through the disklist then realize it can't fit a DLE onto a > > > tape then load the next [correctly labeled] tape? > > Two things, holding disk and DLE splitting. About both, strongly consider > Deb's comment about keeping DLEs to a reasonable size. > > Amanda may be backing up multiple DLEs simultaneously -- if you use an inter- > mediate random access media called a "holding disk". If amanda goes directly > to tape, it can only do one DLE per tape drive at a time. There are many > advantages to using a holding disk, but it has to be quite large, capable of > holding several complete DLEs. A DLE going to disk will not go to tape until > it is complete on the HD. > > Amanda can split DLEs across multiple tapes. See "allow-split" directive in > amanda.conf(5). The DLE is split into chunks enroute to the tape. If a > chunk doesn't fit the current tape, it is sent to the next tape. You specify > the chunck size considering that the average "wasted, unused piece of tape at > the end" will be 1/2 the chunck size. So, perhaps 1,2,5, or 10% of tape > capacity. > > > > I will be loading 8 LTO6 tapes into the tape changer (currently max > > > capacity). I only have this one tape changer at my disposal. > > > > > > I'm looking to do two things with the same data: > > > > > > 1) perform a one-off archive and remove the tapes for safe storage > > > off-site > > > 2) load a new set of tapes and use amanda as a near-line storage > > > solution (quick backup/restore over network) > > > > It sounds like you are thinking of a single full backup (your off-site > archive) and then just backing up the changes thereafter. That is the way > some backup systems operate. Amanda was not designed to work that way though > it is flexible enough that you could coerce it. Recognize that means there > is only a single copy of unchanged files (heaven forbid something happens to > it) and that one copy is off-site, not available for quick restoration. > > > > Both of these can be achieved with the identical config I think? I > > > just need to specify the exact days for 1) then remove the tapes... > > > before doing so, will i need to use amadmin to say do not re-use > > > these tapes? (incase I retrieve from safe storage to do a restore > > > from) > > > > > > Does this sound sensible? > > > > > > dumpcycle 1 week > > > runspercycle 5 > > > tapecycle 8 ? > > These are parameters to be nailed down after you decide on a scheme. > Also "runtapes" is important in that discussion. > > > > > > cronjob to invoke amdump once sun-thur inclusive at night ... > > > any new DLE configured fri, meaning full backup of new DLE performed > > > sunday > > Another "traditional" scheme is full dumps all on one night (Sun?) and > incrementals each other run. Again, amanda can be coerced into this scheme, > but it was not designed for that scheme. > > Amanda does full dumps of individual DLEs on different days. Its design > tries to backup about the same amount of data each run of a dumpcycle. > Using your above parameters, that would mean about 20% of a full dump each > run (plus incrementals). Some runs will do full dumps of 10 DLEs other runs > just 1 big DLE. Because of its design, new DLEs can be added at any time. > They will get their first dump as a full dump the next run. > > Here are schemes two former clients used (still use to my knowledge). > > 1. For legal requirements, some data needed long term archival storage. > Most of the data required recent backup only. This was handled by separate > configs. The long term stuff done "manually" to all new tapes. > At first, once a month, then once a week. Eventually even the archive stuff > split into two configs, some done weekly, others daily. > > The second original config ran on a 2 week dumpcycle and included the data in > the archival config(s) plus much more. > > 2. This client had no need for archival storage but did require disaster > coverage of recent data off-site. They ran a 1 week dumpcycle and their > tapecycle was large enough for 4 dumpcycles in 4 groups. Three groups were > kept on-site, last weeks group, the group in current use, and the 3 week old > group that would be used next week. When that last group became the > "current" group, the first group went off-site, and the off-site group of > tapes came back. > > As you consider your scheme, also consider if your "near-line storage" > needs taping. Amanda can use hard disk as "Virtual Tapes". NAS or some kind > of RAID box could be used. The virtual changer could hold as many tapes as > you want and could be expanded as needed. The LT06 could then be reserved > for a separate configuration intended for off-site archival of periodic full > dumps of everything. All the VTs used for daily runs would be in your > virtual changer available at all times. No physical tape changing. > > HTH, > Jon > -- > Jon H. LaBadie [email protected] > 11226 South Shore Rd. (703) 787-0688 (H) > Reston, VA 20190 (703) 935-6720 (C) -- Jean-François Malouin | IT Operations and Infrastructure McConnell Brain Imaging Centre | MNI | McGill University
