Thomas, Regarding work-area and tapes...
Amanda gets its parallelism by running multiple dumpers simultaniously, multiple DLEs being dumped concurrently to the work area, as each DLE completes it is flushed to disk (excepting of course if you are using tape tuning parameters but we will disregard them and stick to classic behavor for now). If you are dumping directly to tape or vtape, because you lack a work area you are forcing amanda to run the dumps in a non-parallelized mannor, esentially as would occur if you where simply running manual TAR or DUMP commands one after the other (except of course that amanda is setting the dump levels for you). If you have a 'sufficient' work area you will run multiple dumps and you will see in your amanda reports that dump time will be some multiple of the run or wall clock time. Currently I imaging those values are equal 1:1. You will also see that the time to dump each partition and the time to tape each partition will no long be the same, since you can push a work-area file to tape (or vtape) a lot more quickly than you can dump the file to the work area. This is where your real savings will be, time-wise. The work area will not be used if the DLE is larger than the dump area, if you have room for more than one dump in the work area you will begin to have your parallelism. I DO NOT KNOW, if the new version of amanda which can write multiple tapes at the same time will allow multiple dumps directly to vtapeS. Still, even if it does it would result is an effective dump limit of the number of vtapes you are willing to commit to each amanda run. As far as DLE space and work area, any specific DLE is restricted to total work area space, if you are using "chunksize", ie that is the DLE is written not as a single file to the work area but as multiple (perhaps numerous) smaller files (I set my chunksize as 1 Gig) allowing any large DLE to span multiple work areas. As long as you have room to pre-allocate to a DLE based on its size estimate you can allow amanda to start another dump thread. Hope this helps... Brian On Mon, Oct 25, 2010 at 10:26:42AM +0200, Thomas Marko wrote: > Thank you Jon, Brian and Charles for your answers! > > Am 21.10.10 21:50, schrieb Jon LaBadie: > > I'm not sure if it is best to make up 5 distinct Amanda configs, or just > > one config with customized DLEs and dumptypes. I'd probably try the > > latter first. I think it would make the best use of your storage media > > rather than targeting specific amounts for each config. > > First I started with just one configuration, but the time to backup the > data took more than a day. > > My thought was, that the time a backup takes, can be shortened when I > split the data up into parts which should be backuped daily (this data > changes fast) and data that does not change und thus be backuped weekly. > > But if I can accomplish that the duration of a backup run takes not that > long and I can backup all of my data on a daily basis I would be very > happy :-) > > What do you think about the following considerations: > > > You don't seem to be using a holding disk so you will be writing > > directly to the vtapes. That increases the likelyhood of trashing > > something on problems. It also eliminates dumping multiple DLEs > > in parallel. Rather one after the next will have to run. > > - I have 100G on another partition I can use for a holdingdisk. I will > try to keep the DLEs smaller than 100G to achieve that they fit to the > holdingdisk. I will not be able to fit all of them to the holding disk > size (is this a problem?), but the DLEs which are larger should be > dumped directly to the vtapes, the DLEs which fit will use the holding > disk, right? > > - Use a vtape length of 108750 MBytes (= 25 * 4350 (= sizeof a DVD, just > if I would want to archive them to DVDs, but honestly I don't think that > I will do that ever)) > tapelength 108750 Mbytes > > - Split this tapes into DVD sized parts: > tape_splitsize 4350 mbytes > > - dumpcycle 7 days (minimum 1 full dump per week) > > - runspercycle 7 (run daily) > > - tapecycle 16 (= 16 * 108750 Mbytes (=tapelength) = 1740000 Mbytes > (fits on my tapes-partition (1782415 Mbytes aprox. 1.8T) > > - runtapes 2 (needed as the largest DLE is 180G) > > If my config is completely stupid what would your configuration be? > > Should I use smaller or larger vtapes, thus more or less vtapes and > different splitsizes? > Is the holdingdisk in this configuration usefull? > > > Even if you want Daily and Weekly to be different configs, rather > > than DLEs of one config, consider putting the DLEs from the four > > weekly configs into a single config. First benefit you get is > > 24 tapes of the current size. The dumptype could be set to > > "strategy nofull" (or is it "strategy incronly" ?) and use > > amadmin "force" commands (from cron) to specify when to do fulls. > > It's not that important to know when amanda is doing fulldumps (I think > amanda is much cleverer than me to decide when to do it ;-), but my goal > is to keep backup times and the vtape usage as small as possible and > reasonable. > > Will amanda be that "clever" to make the full dumps of each DLE on > different days? If not, is it possible to achieve this? > > When I setup this new configuration and run it initially amanda will > make full dumps of all DLEs in the first run, right? > > Should I run this first backup manually (I think it'll take more than a > day) and after that set up the cron job? > > When using a holding disk and amanda can dump multiple DLEs at a time, > will it be that clever to use DLEs from different hosts to do a bit load > balancing (I do the compressing on client side)? If not, is it possible > to achieve that? > > > Seriously consider moving to the 3.x versions. > > At the moment I would be happy to understand how amanda works and get > amanda running in a way I want it to. I do not want to mess up in a > selfcompiled installation and run into more troubles needed. > > > If you run your backups on the "ragged edge" of sufficient storage, > > I would say your (i.e. your mgmt) do not value highly the data > > that is being backed up. An extra pair of 1-2TB drives would cost > > about $125-$250. Isn't extra safety of your data worth it? > > You are completely right, but there is no management (maybe but my wife > ;-) and the problem is not the amount of $ it's the amount of SATA > controllers ;-) in my backup "server" ;-) > > Am 22.10.10 00:48, schrieb Charles Curley: > > I set things up for a 15 day dump cycle, M-F, so three weeks. This > > implies *at least* one level 0 per DLE. You will find Amanda staggers > > them so as to even out the load. That's much better than one can do > > manually, especially as DLE sizes change over time. Also, Amanda will do > > more than one level 0 per cycle if it finds the assets to do so. > > As the time to backup takes more than a day, for me this is not very > good, as more L0s means longer backup time.. > > Am 21.10.10 22:11, schrieb Brian Cuttler: > > The tape savings, the, don't waste a lot of tape on static > > files is handled for me by the dump levels as scheduled > > by the amanda server. [Discusson how how it does that > > withheld for another time, see 'estimate phase'] > > When I look at my DailySet1 backups for the last days, there are many > backups with the same level. I thought that for example the DLEs > "KonfigFiles" should be backuped once per dumpcycle at L0 and the other > times just incremental as this data _never_ changes. So IMHO it is not > necessary to backup it each day (as it does not change). But obviously I > missed the target here :-) > > The problem are not the small DLEs (they do not take very long to > backup), the problem are the larger ones. If the large ones do not > change, amanda will make L0s of them each time and this can take a very > long time each day. Is it possible to force amanda to change that > behaviour (read something about "strategy nofull" / "strategy > incronly")? Does that make sense? > > Assuming a DLE is 100G large. The data does not change. I have 10*100G > tapes. I have a dumpcycle of 14 and Amanda choose to make a L0 each day, > because the data does not change. What will happen: Will amanda > overwrite the oldest L0 after 10 days? Or will amanda break and tell me > that it ran out of tapes? This is just hypothetical, maybe it helps me > to understand how amanda works... > What if one day the data changes slightly? Will amanda run a L1 in the > next run? What will amanda do after the L1? > > Btw. does in my amoverview DLE hochschw RootHome mean, that amanda made > on 20.-22.10. always incrementals or are incrementals only made when > there is a switch from 0 to 1 and 1 to 2 and so on, so that amanda made > fulls on 19., then made an incremental on 20. and again fulls on 21. and > 22. What happened on 23.? Has the dumpcycle been over and amanda > switched back to L0 and made a full again? > > r...@store02:~# su backup -c "amoverview DailySet1 -skipmissed" > > date 10 10 10 10 10 10 10 10 > host disk 18 19 20 21 22 23 24 25 > > calendar Kalender 0 0 0 0 0 0 0 > calendar KonfigFiles 0 0 0 0 0 0 0 > calendar Roothome 0 0 0 0 0 0 0 > hochschw Backups 0 0 0 0 0 0 0 > hochschw KonfigFiles 0 0 0 0 0 0 0 > hochschw Mailserver 0 0 0 0 1 1 1 > hochschw RootHome 0 1 1 1 0 0 0 > riegerin KonfigFiles 0 0 0 0 0 0 0 > riegerin RootHome 0 0 0 0 0 0 0 > store01. Dokumente 11 11 11 11 11 11 11 11 > store01. KonfigFiles 0 0 0 0 0 0 0 > store01. RootHome 0 0 0 0 0 0 0 > store01. Scratch 1 1 1 1 0 00 00 > store01. WindowsHomes 11 11 11 11 11 11 11 11 > store02. KonfigFiles 0 0 0 0 0 0 0 > store02. RootHome 0 0 0 0 0 0 0 > > Thank you very much for your help! > > Cheers, > Thomas --- Brian R Cuttler [email protected] Computer Systems Support (v) 518 486-1697 Wadsworth Center (f) 518 473-6384 NYS Department of Health Help Desk 518 473-0773 IMPORTANT NOTICE: This e-mail and any attachments may contain confidential or sensitive information which is, or may be, legally privileged or otherwise protected by law from further disclosure. It is intended only for the addressee. If you received this in error or from someone who was not authorized to send it to you, please do not distribute, copy or use it or any attachments. Please notify the sender immediately by reply e-mail and delete this from your system. Thank you for your cooperation.
