> On Nov 16, 2015, at 3:57 PM, David Simpson <[email protected]> wrote:
>
> Hi Deb
>
> Thanks for your help.
>
> I'm still trying to digest it all - being new to Amanda. I'm not sure I'm
> thinking the "Amanda way" with the dumpcycle,runspercycle,tapecycle,runtapes
> yet... but I'm getting closer.
>
> "copy" was the wrong word and should be substituted with "archive" in that
> sentence.
>
> ---------------
>
> Archive
>
> Yes I'm hoping to leave it fully loaded to do its thing with a set of 8 tapes
> and try to minimize human interaction/changing. For the full amount of data I
> will of course need to do some unloading and loading of tapes, but I'd hope
> to remove 8 and replace with 8. Will I end up with a massive amount to flush
> and therefore the holding disk will run out? Maybe I need to split it into 3
> or 4 disklists!
Are you assuming it will take 8 tapes to hold all of your data one time?
[Pause while I look up the capacity of an LTO6 … yes, I guess you are.]
In that case, the AMANDA way of doing things is more like this:
You DO have to divide your data into smaller chunks, called DLEs. I like
chunks that are smaller than 1/4 of a tape, so that amanda can shuffle them
around. THAT’s what she is good at.
So you define a list of DLEs that, in total, add up to all of your data,
but with each DLE being a little less than 1/4 of a tape (fi you take my
advice; others will chime in here.)
That way, no DLE will have to be split across a tape (this makes for easier
restoring to Dead Iron, with no software).
If you have to use TAR and define things as “all the A* files or the { A*,
the a*, the B*, the b*, and the C* } you can do that. HOW will follow
later, or you can look it up.
THEN:
If it’s going to take 8 tapes to get it all on tape, then you either have to
fit 8 runs within each week, do more than 1 tape at each run, or you can’t
say that “dumpcycle” is a week.
If it is actually a tape CHANGER, can it mount the tape itself? Then let it
use 2 of them each night. Yes, you’ll have to change the tapes twice a week,
but you would get all your data
backed up within a week. Or, call your dumpcycle “2 weeks”.
EITHER:
dumpcycle 1 week
runspercycle 5 (or 7 if you can manage to)
runtapes 2 ( or even more tapes each day)
OR:
dumpcycle 2 week
runspercycle 10 (or 14 if you can manage runs on the weekend)
runtapes 1 (if you can only do 1 tape per day) , but more would help you
out.
“dumpcycle’ will tell it how often it has to do a level 0 of each DLE.
You don’t decide which day it does that level 0. They are spread around among
all the DLEs you have defined, so that the data load is approximately equal
each day.
But amanda DOES promise to get at least one level 0 of each DLE within
“dumpcycle” time.
(SOmetimes it “promotes” DLEs and does a level 0 earlier, while it is
shuffling them around, to do that balancing I mentioned.)
Oh, each day it will touch each DLE — but it does level 1 or 2 or etc
backups on the off days.
UNLESS you tell it (and me) you only care about level 0 backups. Then we
start talking about a whole different strategy. Maybe that is more what you
are saying?
If you really don’t EVER intend to overwrite tapes, and always plan to use new
tapes, archiving the first tapes, then say
tapecycle 999
and amanda won’t expect to use the first ones again.
Tapecycle just tells it how many tapes there are all told. Dumpcycle,
runspercycle, and runtapes tell it how often it must do a level 0 of each DLE
and how many tapes it must use each day
to do so.
Does this help a little more? I do think you are going to have to do the
work of dividing up your data. Even if you tell me you only want level 0
backups.
Deb
>
> Just trying to get my head round the 999/infinite tape idea. What would the
> dumpcycle be?
>
> Could you do dumpcycle 0, runspercycle 1 and tapecycle 999 tapes - as in full
> backup and carry on until you run out of tapes. That could work well for the
> archive if I'm understanding correctly.
>
> We'd also need a runtapes number defined. 1 will not do it. If I understand
> correctly anything above 1 results in spanning and will need the tape
> definition plumbed in as you say for Amanda to know the length. Could
> runtapes be 999 as well?
>
> Infinite tapes per run and infinite available!
>
> ---------------
>
> Backup
>
> Beyond this I hope to start a weekly backup with working with less than that
> full volume of data and possibly constrained to 8 tapes.
>
> ---------------
>
> David
>
>
> --------------------
> David Simpson - Computing Support Officer
> IBME - Institute of Biomedical Engineering
> Old Road Campus Research Building
> Oxford, OX3 7DQ
> Tel: 01865 617697 ext: 17697
>
> ________________________________________
> From: Debra S Baddorf [[email protected]]
> Sent: 16 November 2015 18:07
> To: David Simpson
> Cc: Debra S Baddorf; Amanda Users
> Subject: Re: archive and near-line usage
>
>> On Nov 11, 2015, at 3:39 PM, Debra S Baddorf <[email protected]> wrote:
>>
>>
>>> On Nov 11, 2015, at 2:55 PM, David Simpson <[email protected]>
>>> wrote:
>>>
>>> Hi all,
>>>
>>> Thanks for the help so far. This has saved me a lot of time.
>>>
>>> I have a fair amount of data I'm looking to copy (potentially ~30TB) - that
>>> is data from multiple file-servers to multiple tapes and without LEOM. Am I
>>> correct in saying this is the implication(?) - Amanda will get so far
>>> through the disklist then realize it can't fit a DLE onto a tape then load
>>> the next [correctly labeled] tape?
>
> I’ve got a question about THIS STEP — see below.
>
>
>>>
>>>
>>> I will be loading 8 LTO6 tapes into the tape changer (currently max
>>> capacity). I only have this one tape changer at my disposal.
>>>
>>> I'm looking to do two things with the same data:
>>>
>>> 1) perform a one-off archive and remove the tapes for safe storage off-site
>>> 2) load a new set of tapes and use amanda as a near-line storage solution
>>> (quick backup/restore over network)
>>>
>>> Both of these can be achieved with the identical config I think? I just
>>> need to specify the exact days for 1) then remove the tapes... before doing
>>> so, will i need to use amadmin to say do not re-use these tapes? (incase I
>>> retrieve from safe storage to do a restore from)
>>>
>>> Does this sound sensible?
>>>
>>> dumpcycle 1 week
>>> runspercycle 5
>>> tapecycle 8 ?
>>
>> To address only THIS part of the question: I would recommend a MINIMUM of
>> tapecycle 15 # 3 weeks - 5 runs times 3 weeks.
>> and if you can manage it
>> tapecycle 20 # 4 weeks - 5 runs times 4 weeks.
>> or more. If you use one tape per night (runtapes 1 is the default;
>> you didn’t mention how many you need?) then you are only giving yourself
>> half a week before you overwrite your last full backup. 8 tapes means you
>> are overwriting a full backup BEFORE you write another full backup,
>> and you might not be able to do a restore. Amanda doesn’t promise on WHICH
>> day any one DLE will get a full done. So maybe the DLE
>> that you need to restore today doesn’t have ANY full because you overwrote
>> it last night? (Yeah, I just said that, didn’t I.)
>>
>> My philosophy is “if the data is worth anything at all to us, then it is
>> worth buying enough tapes to have at least 4 weeks of backups”.
>> I’ve got a 10 week rotation of tapes right now.
>>
>> Deb Baddorf
>> Fermilab
>>
>
> With no other replies, I’ve re-read and will assay another answer:
> If you are using a DIFFERENT set of ~8 tapes every week, then why not just
> TELL amanda that you have 999 tapes (or some near infinite number).
> Then you don’t have to worry about issuing the “no re-use” commands, I
> don’t think, because it won’t WANT to re-use any tapes, until you hit the
> 999 number (in which case, pick a bigger number yet).
>
> Question about your COPY step:
>
> What kind of “copy” do you envision? If you just do an amanda
> configuration to backup those DLE’s, amanda will try to guess correctly
> about the size of the last DLE and making it fit onto the tape, BUT that is
> dependent on your correct number for the tape size. As I understand it,
> she will then try another DLE too (a small one, since that’s what size
> still seems to be left) and when THAT doesn’t fit, that DLE will not have
> a LEOT and will be redone on the next tape.
>
> If you are using a non-amanda copy of individual DLEs, then amanda won’t
> have anything to do with your resultant tape. But you can use amanda’s
> AMVAULT command to copy from one set of media to another. Perhaps that’s
> what you need? I’ve done some of this ….. copying LTO2 tapes to virtual
> disk “media” and then, on another machine, copying from those virtual disk
> media to LTO5 tapes. That worked well, once I got the commands right.
>
> Sorry if you’ve already gotten this to work, but since nobody else answered,
> I thought I’d try again.
> Deb Baddorf
> Fermilab
>
>
>>
>>
>>>
>>> cronjob to invoke amdump once sun-thur inclusive at night ... any new DLE
>>> configured fri, meaning full backup of new DLE performed sunday
>>>
>>>
>>> thanks
>>> David
>>>
>>> --------------------
>>> David Simpson - Computing Support Officer
>>> IBME - Institute of Biomedical Engineering
>>> Old Road Campus Research Building
>>> Oxford, OX3 7DQ
>>> Tel: 01865 617697 ext: 17697
>>
>
