Hi Deb, Jon
Hopefully this will cover both responses. Thanks again...
Holdingdisk is already configured as 2000GB that as much as I can do on that
server.
Chunksize I have already set to 500MB
Advice on 1/4 of tape size for max DE noted.
>>Archive - with infinite tapes: Amanda will chug through all the tapes in
>>your changer, and then will wait until you put some more new tapes in there
(note to self: no it won’t, it will fail cuz it can’t find tapes. How can
we tell it to WAIT ON THE OPERATOR to change tapes?)
Interested in this. Will the dump fail? What will happen will it stop with a
DLE in the Holdingdisk waiting to be flushed?
----------------------
I ran the tape profiler and got these numbers seen in my definition (I also
added part size into the definition). I don't know why I didn't get closer to
170mps or 180mps.... kernel module/driver?
define tapetype "HP-Ultrium-LTO6" {
comment "Created by amtapetype; compression enabled"
length 2299845440 kbytes
speed 69251 kps
blocksize 32 kbytes
#use default filemark - don't specify
#filemark
#needed for spanning... set low to try to avoid wastage (in abscence of
LEOM)
part_size 100G
}
-------------------------
>>It sounds like you are thinking of a single full backup (your off-site
>>archive) and then just backing up the changes thereafter. That
-I'm planning on a full backup (off site archive).
-After that a full backup and proper cycle, likely a weekly or fortnightly
dumpcycle (so Amanda running in a more typical fashion) - but to do this... I'm
going to make a call on what I will and won't backup to try to reduce the
volume (probably to 7-8 tapes worth max). As Deb importantly point outs.
>> Using your above parameters, that would mean about 20% of a full dump each
>> run (plus incrementals).
An important point. Perhaps it would be prudent to change to this (?) to reduce
the nightly volume:
dumpcycle 2 weeks
runspercycle 10
tapecycle 8
I would want to avoid spilling into the daytime (7 or 8am) with the regular
backup if possible.
regards
David
--------------------
David Simpson - Computing Support Officer
IBME - Institute of Biomedical Engineering
Old Road Campus Research Building
Oxford, OX3 7DQ
Tel: 01865 617697 ext: 17697
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Jon LaBadie
Sent: 18 November 2015 06:01
To: Amanda Users <[email protected]>
Subject: Re: archive and near-line usage
On Tue, Nov 17, 2015 at 11:27:30PM +0000, Debra S Baddorf wrote:
> AMANDA USERS LIST: anybody else want to jump in here with better ideas??
>
> I’m open to better suggestions ….. I’m probably making a real hash of
> this, and
> not necessarily making suggestions in the best order…..
> Deb
I'll throw in a few thoughts.
>
> > On Nov 11, 2015, at 2:55 PM, David Simpson <[email protected]>
> > wrote:
> >
> > Hi all,
> >
> > Thanks for the help so far. This has saved me a lot of time.
> >
> > I have a fair amount of data I'm looking to copy (potentially ~30TB)
> > - that is data from multiple file-servers to multiple tapes and without
> > LEOM.
> > Am I correct in saying this is the implication(?) - Amanda will get
> > so far through the disklist then realize it can't fit a DLE onto a
> > tape then load the next [correctly labeled] tape?
Two things, holding disk and DLE splitting. About both, strongly consider
Deb's comment about keeping DLEs to a reasonable size.
Amanda may be backing up multiple DLEs simultaneously -- if you use an inter-
mediate random access media called a "holding disk". If amanda goes directly
to tape, it can only do one DLE per tape drive at a time. There are many
advantages to using a holding disk, but it has to be quite large, capable of
holding several complete DLEs. A DLE going to disk will not go to tape until
it is complete on the HD.
Amanda can split DLEs across multiple tapes. See "allow-split" directive in
amanda.conf(5). The DLE is split into chunks enroute to the tape. If a chunk
doesn't fit the current tape, it is sent to the next tape. You specify the
chunck size considering that the average "wasted, unused piece of tape at the
end" will be 1/2 the chunck size. So, perhaps 1,2,5, or 10% of tape capacity.
> > I will be loading 8 LTO6 tapes into the tape changer (currently max
> > capacity). I only have this one tape changer at my disposal.
> >
> > I'm looking to do two things with the same data:
> >
> > 1) perform a one-off archive and remove the tapes for safe storage
> > off-site
> > 2) load a new set of tapes and use amanda as a near-line storage
> > solution (quick backup/restore over network)
> >
It sounds like you are thinking of a single full backup (your off-site archive)
and then just backing up the changes thereafter. That is the way some backup
systems operate. Amanda was not designed to work that way though it is
flexible enough that you could coerce it. Recognize that means there is only a
single copy of unchanged files (heaven forbid something happens to it) and that
one copy is off-site, not available for quick restoration.
> > Both of these can be achieved with the identical config I think? I
> > just need to specify the exact days for 1) then remove the tapes...
> > before doing so, will i need to use amadmin to say do not re-use
> > these tapes? (incase I retrieve from safe storage to do a restore
> > from)
> >
> > Does this sound sensible?
> >
> > dumpcycle 1 week
> > runspercycle 5
> > tapecycle 8 ?
These are parameters to be nailed down after you decide on a scheme.
Also "runtapes" is important in that discussion.
> >
> > cronjob to invoke amdump once sun-thur inclusive at night ...
> > any new DLE configured fri, meaning full backup of new DLE performed
> > sunday
Another "traditional" scheme is full dumps all on one night (Sun?) and
incrementals each other run. Again, amanda can be coerced into this scheme,
but it was not designed for that scheme.
Amanda does full dumps of individual DLEs on different days. Its design tries
to backup about the same amount of data each run of a dumpcycle.
Using your above parameters, that would mean about 20% of a full dump each run
(plus incrementals). Some runs will do full dumps of 10 DLEs other runs just 1
big DLE. Because of its design, new DLEs can be added at any time. They will
get their first dump as a full dump the next run.
Here are schemes two former clients used (still use to my knowledge).
1. For legal requirements, some data needed long term archival storage.
Most of the data required recent backup only. This was handled by separate
configs. The long term stuff done "manually" to all new tapes.
At first, once a month, then once a week. Eventually even the archive stuff
split into two configs, some done weekly, others daily.
The second original config ran on a 2 week dumpcycle and included the data in
the archival config(s) plus much more.
2. This client had no need for archival storage but did require disaster
coverage of recent data off-site. They ran a 1 week dumpcycle and their
tapecycle was large enough for 4 dumpcycles in 4 groups. Three groups were
kept on-site, last weeks group, the group in current use, and the 3 week old
group that would be used next week. When that last group became the "current"
group, the first group went off-site, and the off-site group of tapes came back.
As you consider your scheme, also consider if your "near-line storage"
needs taping. Amanda can use hard disk as "Virtual Tapes". NAS or some kind
of RAID box could be used. The virtual changer could hold as many tapes as you
want and could be expanded as needed. The LT06 could then be reserved for a
separate configuration intended for off-site archival of periodic full dumps of
everything. All the VTs used for daily runs would be in your virtual changer
available at all times. No physical tape changing.
HTH,
Jon
--
Jon H. LaBadie [email protected]
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (703) 935-6720 (C)
