On Tue, Mar 04, 2014 at 05:16:05PM -0500, Michael Stauffer wrote: > On Fri, Feb 28, 2014 at 1:07 AM, Jon LaBadie <[email protected]> wrote: > > > On Thu, Feb 27, 2014 at 07:19:24PM -0500, Michael Stauffer wrote: > > > Amanda 3.3.4 > > > > > > Hi, > > > > > > Another long post from me - thanks to anyone who has time to read it. > > > > > > I've been reading various docs and posts about dumpcycle, runspercycle, > > > tapecycle, runtapes online, but still can't figure out how I should set > > > things up for my needs. > > > > > > I've got: > > > - ca. 30TB of data to backup > > > - I'll try to split this into 500GB or 1TB DLE's, but some will be much > > > easier to leave as 2-4TB DLE's > > > - I'm fine with a level 0 only every 30 days, especially considering a > > full > > > backup will take 8-10 days. > > > - I've got 1.5TB tapes that hold about 2TB of data with hardware > > > compression from testing > > > - my changer holds 35 tapes > > > > First question, will you be using more than 35 tapes. I.e. will you > > periodically pull some recently used and replace with less recently > > used tapes? If not, I think you are short on tapes. A full dump will > > take 15 tapes. You really want a MINIMUM of two full dumps. I prefer > > more. So 2x15 is two full dumps leaving only 5 tapes for incrementals. > > I don't think that will be enough. > > > > I was thinking of doing a periodic (at the start and then every 3 months) > level 0 archive dump to a different set of tapes (probably four sets to > retain 1 year's worth of dumps). Then I thought the library in the changer > would be fine if it held less than two level 0 dumps at any time. I'd > rather just go switch tapes once every 3 months than more often, and have > offset archive too. Does that seem reasonable? > I wouldn't be concerned about how many currently in the library. I'd consider how many total I have. Anyone who has administered tape backups for any significant time has a war story. Tapes that seemed to write correctly but a month later were not readable. Damage, physical and environmental. My own involves a large magnet that I did not realize was right next to my tape storage.
If you only have one level 0 and it goes bad, you have no backup! > > > > > I'd like to have the changer always hold a level 0 dump and then the set > > of > > > subseuqent incrementals. So it should take about 15-20 tapes for a level > > 0 > > > of all DLE's, and then the incrementals over a month should easily fit > > > within 2-4 tapes, judging from experience here. When the next level 0 > > dump > > > starts, I'd like amanda to use as many of the remaining 10 or so tapes > > > before overwriting tapes from the old level 0 dump (overwriting only > > tapes > > > of DLE's that have just had a new level 0, of course). (I will > > periodically > > > do a level 0 dump to a different tape set for offset archiving) > > > > > Do you plan to let amanda do the scheduling? Or are you going to force > > her kicking and screaming into the traditional schedule of one monster > > full dump followed by all incrementals. Then another monster. Blech. > > > I'm fine with Amanda's scheduling. When I do my first round of amdump's > though, it will be effectively a monster dump until all DLE's have a level > 0. Then I presume amanda can even things out using her own scheduling? > You can ease amanda into your dumpcycle. Add a couple of DLEs with each dump run until they are all added. Avoids the initial monster dump. > > > How do I setup dumpcycle, runspercycle, tapecycle, runtapes to achieve > > this? > > > > > > >From the docs, it seems I'd want: > > > dumpcycle 30 days > > > runspercycle 15 #15, for running amdump every other day > > > runtapes 2 # to allow for DLE's that can get up to 4TB > > > tapecycle 34 # at least (runspercycle + 1) * runtapes - per docs > > > suggestion > > > # and leave one extra as a spare > > > > > > Is this right? > > > > > > Does this mean that when I run amdump, it will at most write two > > > tapes-worth of DLE's, and then stop? Then the next run will pick up from > > > there? I think so, but would like to make sure. I'm used to the manual > > > paradigm of "run a full backup" and then do incrementals. But this seems > > > that it will level out to be the same in the end as that? > > > > > > HOWEVER, I'd rather have runtapes at 3 or 4 to minimize tape waste and > > make > > > it less critical to have evenly-sized DLE's, which will be difficult to > > > maintain. But if runtapes is 3, the recommended value of tapecycle would > > be > > > >= 48, more than my # of tapes. But in practice, 35 should still plenty > > of > > > tapes to do what I want without overwriting level 0's prematurely. It > > seems > > > like tapecycle minimum should be more like '# of tapes per full backup + > > # > > > of tapes for incrementals over dumpcycle + 2 * runtapes', plus one or two > > > as a buffer. > > > > The following sentence shows you are still thinking the "non-amanda way. > > > > > The formula in the docs of "(runspercycle + 1) * runtapes" > > > plays it very safe when you consider many incremental dumps will go to > > > holding disk and be collected onto one tape periodically. > > > > There are not going to be a bunch of incrementls to collect onto one > > tape. Each amdump run will be a mix of level 0's for some DLEs and > > incrementals for the others. > > > > With 15 runspercycle you will AVERAGE 2TB of level 0 plus ??GB of > > incrementals. But remember you said some DLEs will be 4 or more TB. > > When they get level 0's you'll need more than 2 tapes. > > > Can I be sure that there will always be some level 0's in a run of amdump? > I guess because there's ~30TB of data and 15 runs, so that pretty much > guarantees a level 0 every run? If not, or if they're << tape-size, I'd > like them to go to holding disk so as not to greatly underutilize a tape. > I'm going to try to have my largest DLE < 2TB - there will be 3 or 4 of > those probably - for simplicity's sake. Depends on the number and size of your DLEs and the dumpcycle. > However it's not clear, but it seems like when using a holding disk the > whole DLE is written to holding disk before streaming to tape, which seems > inefficient time-wise when the DLE is large enough that's going to get > streamed to tape anyway. Most experience the opposite. Taping is considerably faster than dumping. So by dumping directly to tape you are streaming to tape less than optimally. This results in the tape stopping, backing up so that it can get up to speed at the point it needs to start writing again. When you are dumping direct to tape, only one DLE is dumping. If they are going to holding disk, several DLEs can be dumping at the same time. > I guess I have to work params something like this: > > # to have amanda start writing when 10% of volume capacity is written to > holding disk, and > # flush-threshold-scheduled is satisfied > flush-threshold-dumped 10 > > # have amanda write to tape if the holding-disk storage plus planned data > exceeds this > # % of tape capacity > flush-threshold-scheduled 90 > > # taperflush param not needed if above two params are used > #taperflush 90 > > Do I understand right that these above settings will tell amanda to start > streaming to tape relatively early in a dump process if it knows it's going > to end up with >= 90% of volume capacity on the holding disk? I'm thinking > this should make the process more time efficient since it won't write > everything to disk before streaming to tape? Haven't played with these params so have no guidance to offer. jl -- Jon H. LaBadie [email protected] 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (609) 477-8330 (C)
