On Fri, Feb 28, 2014 at 1:07 AM, Jon LaBadie <[email protected]> wrote:
> On Thu, Feb 27, 2014 at 07:19:24PM -0500, Michael Stauffer wrote: > > Amanda 3.3.4 > > > > Hi, > > > > Another long post from me - thanks to anyone who has time to read it. > > > > I've been reading various docs and posts about dumpcycle, runspercycle, > > tapecycle, runtapes online, but still can't figure out how I should set > > things up for my needs. > > > > I've got: > > - ca. 30TB of data to backup > > - I'll try to split this into 500GB or 1TB DLE's, but some will be much > > easier to leave as 2-4TB DLE's > > - I'm fine with a level 0 only every 30 days, especially considering a > full > > backup will take 8-10 days. > > - I've got 1.5TB tapes that hold about 2TB of data with hardware > > compression from testing > > - my changer holds 35 tapes > > First question, will you be using more than 35 tapes. I.e. will you > periodically pull some recently used and replace with less recently > used tapes? If not, I think you are short on tapes. A full dump will > take 15 tapes. You really want a MINIMUM of two full dumps. I prefer > more. So 2x15 is two full dumps leaving only 5 tapes for incrementals. > I don't think that will be enough. > I was thinking of doing a periodic (at the start and then every 3 months) level 0 archive dump to a different set of tapes (probably four sets to retain 1 year's worth of dumps). Then I thought the library in the changer would be fine if it held less than two level 0 dumps at any time. I'd rather just go switch tapes once every 3 months than more often, and have offset archive too. Does that seem reasonable? > > > I'd like to have the changer always hold a level 0 dump and then the set > of > > subseuqent incrementals. So it should take about 15-20 tapes for a level > 0 > > of all DLE's, and then the incrementals over a month should easily fit > > within 2-4 tapes, judging from experience here. When the next level 0 > dump > > starts, I'd like amanda to use as many of the remaining 10 or so tapes > > before overwriting tapes from the old level 0 dump (overwriting only > tapes > > of DLE's that have just had a new level 0, of course). (I will > periodically > > do a level 0 dump to a different tape set for offset archiving) > > > Do you plan to let amanda do the scheduling? Or are you going to force > her kicking and screaming into the traditional schedule of one monster > full dump followed by all incrementals. Then another monster. Blech. I'm fine with Amanda's scheduling. When I do my first round of amdump's though, it will be effectively a monster dump until all DLE's have a level 0. Then I presume amanda can even things out using her own scheduling? > > How do I setup dumpcycle, runspercycle, tapecycle, runtapes to achieve > this? > > > > >From the docs, it seems I'd want: > > dumpcycle 30 days > > runspercycle 15 #15, for running amdump every other day > > runtapes 2 # to allow for DLE's that can get up to 4TB > > tapecycle 34 # at least (runspercycle + 1) * runtapes - per docs > > suggestion > > # and leave one extra as a spare > > > > Is this right? > > > > Does this mean that when I run amdump, it will at most write two > > tapes-worth of DLE's, and then stop? Then the next run will pick up from > > there? I think so, but would like to make sure. I'm used to the manual > > paradigm of "run a full backup" and then do incrementals. But this seems > > that it will level out to be the same in the end as that? > > > > HOWEVER, I'd rather have runtapes at 3 or 4 to minimize tape waste and > make > > it less critical to have evenly-sized DLE's, which will be difficult to > > maintain. But if runtapes is 3, the recommended value of tapecycle would > be > > >= 48, more than my # of tapes. But in practice, 35 should still plenty > of > > tapes to do what I want without overwriting level 0's prematurely. It > seems > > like tapecycle minimum should be more like '# of tapes per full backup + > # > > of tapes for incrementals over dumpcycle + 2 * runtapes', plus one or two > > as a buffer. > > The following sentence shows you are still thinking the "non-amanda way. > > > The formula in the docs of "(runspercycle + 1) * runtapes" > > plays it very safe when you consider many incremental dumps will go to > > holding disk and be collected onto one tape periodically. > > There are not going to be a bunch of incrementls to collect onto one > tape. Each amdump run will be a mix of level 0's for some DLEs and > incrementals for the others. > > With 15 runspercycle you will AVERAGE 2TB of level 0 plus ??GB of > incrementals. But remember you said some DLEs will be 4 or more TB. > When they get level 0's you'll need more than 2 tapes. Can I be sure that there will always be some level 0's in a run of amdump? I guess because there's ~30TB of data and 15 runs, so that pretty much guarantees a level 0 every run? If not, or if they're << tape-size, I'd like them to go to holding disk so as not to greatly underutilize a tape. I'm going to try to have my largest DLE < 2TB - there will be 3 or 4 of those probably - for simplicity's sake. However it's not clear, but it seems like when using a holding disk the whole DLE is written to holding disk before streaming to tape, which seems inefficient time-wise when the DLE is large enough that's going to get streamed to tape anyway. I guess I have to work params something like this: # to have amanda start writing when 10% of volume capacity is written to holding disk, and # flush-threshold-scheduled is satisfied flush-threshold-dumped 10 # have amanda write to tape if the holding-disk storage plus planned data exceeds this # % of tape capacity flush-threshold-scheduled 90 # taperflush param not needed if above two params are used #taperflush 90 Do I understand right that these above settings will tell amanda to start streaming to tape relatively early in a dump process if it knows it's going to end up with >= 90% of volume capacity on the holding disk? I'm thinking this should make the process more time efficient since it won't write everything to disk before streaming to tape? Thanks again -M
