Re: 'labelstr' param in amanda.conf
Deb, thanks very much for this detailed answer. So if I understand correctly, if I follow your advice and have a number of entries in my config's DLE, and call 'amdump myconfig', each DLE entry is handled as a separate dump as amdump runs? That is, my main concern is to do dumps in chunks so that with large full dumps, if there's an error, there's no single massive dump that gets compromised and must be repeated. Also, worst-case scenario restoration from smaller dumps will be easier. Does that sound right? When doing incremental dumps with this setup, does amdump combine the changed files from various entries in the DLE until it reaches the minimum dumpsize I set to fill a tape? Or is each DLE still handled separately even if none of them is large enough to fill a tape (or match whatever dump size limit is set). Also, I like the advice on doing archive dumps, seems straightforward. I'll be needing to do those too. -M On Thu, Jan 16, 2014 at 11:41 PM, Debra S Baddorf badd...@fnal.gov wrote: Since others have diverted into providing answers for your tape type, I'll resurrect this question and give some sort of an answer. Yes, you probably need just one configuration and one group of tapes labelled per 'labelstr'. You create a list of DLEs to make smaller than tarp sized chunks. node /diskname dumpname [spindle Local ] (optional) node /disk2/aDir tar-type-dumpname [...] node /disk2/another tar-type-dumpname [...] node2 /diskname dumpname [spindle Local ] (optional) node2 /disk2/aDir tar-type-dumpname [...] node2 /disk2/another tar-type-dumpname [...] Etc Then, optimally, you create a cron job for each night (or maybe only week nights), something like this: 21 00 * * * amdump myconfig And let Amanda shuffle your DLEs into a similarly sized amount each night. Stop reading here, for the simplest setup. --- If perchance the manager of nodeN insists on checking his dumpdates log on the same day each week, you CAN force nodeN with another cron entry script: If Wednesday then amadmin myconfig force nodeN * Or perhaps only amadmin myconfig force nodeN /usr Things like that. It forces Amanda to do a level 0 THAT day, on whatever DLEs you do it for. It messes up Amanda's balancing, but if you have a lot of DLEs it'll even out somewhat. You also CAN do a force on all DLEs ( * * ) on the day that you want a whole set of level 0's. But that really messes up the balancing isn't the best idea. - I personally DO run a second configuration (archive) and a second set of tapes (same tape drive) so that I can get an archive tape. I set it no record so it doesn't change the dumpdates, and so it doesn't stop other level 0's from being done in the normal configuration . I do a force on all my DLEs and run it once a month. These tapes go offsite, so I want my daily configuration to have its own set of level 0's. Level 0's at least once a week (my dumpcycle) but then I keep 5-6 weeks of tapes before I recycle and start overwriting those level 0's. Deb On Jan 16, 2014, at 5:01 PM, Michael Stauffer mgsta...@gmail.com wrote: Thanks for the reply. I don't have multiple configurations, I'm just trying to figure out how to set things up. So it sounds like I can have a single configuration that uses the same group of tapes as defined by 'labelstr', and I use the DLE lists to break things up into manageable chunks. Then with amdump, I can specify the individual DLE's to be dumped? -M On Thu, Jan 9, 2014 at 1:58 PM, Jean-Louis Martineau martin...@zmanda.com wrote: On 01/09/2014 01:47 PM, Michael Stauffer wrote: Hi, I'm setting up amanda 3.3.4. Regarding 'labelstr' in amanda.conf: The documentation says: If multiple configurations are run from the same tape server host, it is helpful to set their labels to different strings (for example, DAILY[0-9][0-9]* vs. ARCHIVE[0-9][0-9]*) to avoid overwriting each other's tapes. Does this mean that if I have multiple configurations in order to break up a large 30TB data set into managable chunks, each configuration will have to have a particular set of tapes assigned to it? That seems very awkward if so. yes Why do you have multiple configurations? You should do it with one configuration and multiple dle. Jean-Louis
Re: 'labelstr' param in amanda.conf
Yes, each DLE is handled as a separate dump --- which is why specifying the spindle number is good, if several DLE's are on the same spindle. (However, I never specify the spindle number!) Yes, you definitely want your DLE's in chunks. A chunk of half the size of your tape feels right to me. That way, a big level 0 dump can fit on a tape, and a bunch of smaller DLEs or level 1 dumps can still fit on the tape too. If you are using virtual tapes, then just decide what is a reasonable size for each DLE (regarding having to retry if it fails, etc). On Feb 3, 2014, at 2:44 PM, Michael Stauffer mgsta...@gmail.com wrote: Deb, thanks very much for this detailed answer. So if I understand correctly, if I follow your advice and have a number of entries in my config's DLE, and call 'amdump myconfig', each DLE entry is handled as a separate dump as amdump runs? That is, my main concern is to do dumps in chunks so that with large full dumps, if there's an error, there's no single massive dump that gets compromised and must be repeated. Also, worst-case scenario restoration from smaller dumps will be easier. Does that sound right? Yes, to all of these! Just don't subdivide so much that it becomes unwieldy. And, you want to make sure you don't miss any new files, so I always have one dump which is /disk (meaning all of it) but with an Tar exclusion file excluding all the areas that I've already separately dumped (such as /disk/A/disk/B ). That way, if somebody creates an area like /disk/Aa or even /disk/123 it DOES get backed up somewhere, on my AllTheRest DLE. When doing incremental dumps with this setup, does amdump combine the changed files from various entries in the DLE until it reaches the minimum dumpsize I set to fill a tape? Or is each DLE still handled separately even if none of them is large enough to fill a tape (or match whatever dump size limit is set). No it won't combine them, but it will put the small dumps from lots of different DLEs onto the same tape, until the tape gets full. They are still handled separately. Lots of different DLEs go onto the same same wether they are level 0 or incremental backups -- they are just a different *.dmp (and then gzip) file on the tape. The tape gets lots of those, until it is full. Deb Also, I like the advice on doing archive dumps, seems straightforward. I'll be needing to do those too. -M On Thu, Jan 16, 2014 at 11:41 PM, Debra S Baddorf badd...@fnal.gov wrote: Since others have diverted into providing answers for your tape type, I'll resurrect this question and give some sort of an answer. Yes, you probably need just one configuration and one group of tapes labelled per 'labelstr'. You create a list of DLEs to make smaller than tarp sized chunks. node /diskname dumpname [spindle Local ] (optional) node /disk2/aDir tar-type-dumpname [...] node /disk2/another tar-type-dumpname [...] node2 /diskname dumpname [spindle Local ] (optional) node2 /disk2/aDir tar-type-dumpname [...] node2 /disk2/another tar-type-dumpname [...] Etc Then, optimally, you create a cron job for each night (or maybe only week nights), something like this: 21 00 * * * amdump myconfig And let Amanda shuffle your DLEs into a similarly sized amount each night. Stop reading here, for the simplest setup. --- If perchance the manager of nodeN insists on checking his dumpdates log on the same day each week, you CAN force nodeN with another cron entry script: If Wednesday then amadmin myconfig force nodeN * Or perhaps only amadmin myconfig force nodeN /usr Things like that. It forces Amanda to do a level 0 THAT day, on whatever DLEs you do it for. It messes up Amanda's balancing, but if you have a lot of DLEs it'll even out somewhat. You also CAN do a force on all DLEs ( * * ) on the day that you want a whole set of level 0's. But that really messes up the balancing isn't the best idea. - I personally DO run a second configuration (archive) and a second set of tapes (same tape drive) so that I can get an archive tape. I set it no record so it doesn't change the dumpdates, and so it doesn't stop other level 0's from being done in the normal configuration . I do a force on all my DLEs and run it once a month. These tapes go offsite, so I want my daily configuration to have its own set of level 0's. Level 0's at least once a week (my dumpcycle) but then I keep 5-6 weeks of tapes before I recycle and start overwriting those level 0's. Deb On Jan 16, 2014, at 5:01 PM, Michael Stauffer mgsta...@gmail.com wrote: Thanks for the reply. I don't have multiple configurations, I'm just trying to figure out how to set things up. So it sounds like I can have a single configuration that uses the same group of tapes
Re: 'labelstr' param in amanda.conf
Thanks for the reply. I don't have multiple configurations, I'm just trying to figure out how to set things up. So it sounds like I can have a single configuration that uses the same group of tapes as defined by 'labelstr', and I use the DLE lists to break things up into manageable chunks. Then with amdump, I can specify the individual DLE's to be dumped? -M On Thu, Jan 9, 2014 at 1:58 PM, Jean-Louis Martineau martin...@zmanda.comwrote: On 01/09/2014 01:47 PM, Michael Stauffer wrote: Hi, I'm setting up amanda 3.3.4. Regarding 'labelstr' in amanda.conf: The documentation says: If multiple configurations are run from the same tape server host, it is helpful to set their labels to different strings (for example, DAILY[0-9][0-9]* vs. ARCHIVE[0-9][0-9]*) to avoid overwriting each other's tapes. Does this mean that if I have multiple configurations in order to break up a large 30TB data set into managable chunks, each configuration will have to have a particular set of tapes assigned to it? That seems very awkward if so. yes Why do you have multiple configurations? You should do it with one configuration and multiple dle. Jean-Louis
Re: 'labelstr' param in amanda.conf
Since others have diverted into providing answers for your tape type, I'll resurrect this question and give some sort of an answer. Yes, you probably need just one configuration and one group of tapes labelled per 'labelstr'. You create a list of DLEs to make smaller than tarp sized chunks. node /diskname dumpname [spindle Local ] (optional) node /disk2/aDir tar-type-dumpname [...] node /disk2/another tar-type-dumpname [...] node2 /diskname dumpname [spindle Local ] (optional) node2 /disk2/aDir tar-type-dumpname [...] node2 /disk2/another tar-type-dumpname [...] Etc Then, optimally, you create a cron job for each night (or maybe only week nights), something like this: 21 00 * * * amdump myconfig And let Amanda shuffle your DLEs into a similarly sized amount each night. Stop reading here, for the simplest setup. --- If perchance the manager of nodeN insists on checking his dumpdates log on the same day each week, you CAN force nodeN with another cron entry script: If Wednesday then amadmin myconfig force nodeN * Or perhaps only amadmin myconfig force nodeN /usr Things like that. It forces Amanda to do a level 0 THAT day, on whatever DLEs you do it for. It messes up Amanda's balancing, but if you have a lot of DLEs it'll even out somewhat. You also CAN do a force on all DLEs ( * * ) on the day that you want a whole set of level 0's. But that really messes up the balancing isn't the best idea. - I personally DO run a second configuration (archive) and a second set of tapes (same tape drive) so that I can get an archive tape. I set it no record so it doesn't change the dumpdates, and so it doesn't stop other level 0's from being done in the normal configuration . I do a force on all my DLEs and run it once a month. These tapes go offsite, so I want my daily configuration to have its own set of level 0's. Level 0's at least once a week (my dumpcycle) but then I keep 5-6 weeks of tapes before I recycle and start overwriting those level 0's. Deb On Jan 16, 2014, at 5:01 PM, Michael Stauffer mgsta...@gmail.commailto:mgsta...@gmail.com wrote: Thanks for the reply. I don't have multiple configurations, I'm just trying to figure out how to set things up. So it sounds like I can have a single configuration that uses the same group of tapes as defined by 'labelstr', and I use the DLE lists to break things up into manageable chunks. Then with amdump, I can specify the individual DLE's to be dumped? -M On Thu, Jan 9, 2014 at 1:58 PM, Jean-Louis Martineau martin...@zmanda.commailto:martin...@zmanda.com wrote: On 01/09/2014 01:47 PM, Michael Stauffer wrote: Hi, I'm setting up amanda 3.3.4. Regarding 'labelstr' in amanda.conf: The documentation says: If multiple configurations are run from the same tape server host, it is helpful to set their labels to different strings (for example, DAILY[0-9][0-9]* vs. ARCHIVE[0-9][0-9]*) to avoid overwriting each other's tapes. Does this mean that if I have multiple configurations in order to break up a large 30TB data set into managable chunks, each configuration will have to have a particular set of tapes assigned to it? That seems very awkward if so. yes Why do you have multiple configurations? You should do it with one configuration and multiple dle. Jean-Louis
'labelstr' param in amanda.conf
Hi, I'm setting up amanda 3.3.4. Regarding 'labelstr' in amanda.conf: The documentation says: If multiple configurations are run from the same tape server host, it is helpful to set their labels to different strings (for example, DAILY[0-9][0-9]* vs. ARCHIVE[0-9][0-9]*) to avoid overwriting each other's tapes. Does this mean that if I have multiple configurations in order to break up a large 30TB data set into managable chunks, each configuration will have to have a particular set of tapes assigned to it? That seems very awkward if so. -M
Re: 'labelstr' param in amanda.conf
On 01/09/2014 01:47 PM, Michael Stauffer wrote: Hi, I'm setting up amanda 3.3.4. Regarding 'labelstr' in amanda.conf: The documentation says: If multiple configurations are run from the same tape server host, it is helpful to set their labels to different strings (for example, DAILY[0-9][0-9]* vs. ARCHIVE[0-9][0-9]*) to avoid overwriting each other's tapes. Does this mean that if I have multiple configurations in order to break up a large 30TB data set into managable chunks, each configuration will have to have a particular set of tapes assigned to it? That seems very awkward if so. yes Why do you have multiple configurations? You should do it with one configuration and multiple dle. Jean-Louis
Re: 'labelstr' param in amanda.conf
op 09-01-14 19:47, Michael Stauffer schreef: Does this mean that if I have multiple configurations in order to break up a large 30TB data set into managable chunks, each configuration will have to have a particular set of tapes assigned to it? That seems very awkward if so. Why would you make multiple configurations for that? Gerrit
