Re: Overlapping backups: should I expect problems?
Dustin J. Mitchell wrote: On Mon, Sep 15, 2008 at 11:36 PM, John Morris [EMAIL PROTECTED] wrote: My question is, is this actually a good idea? Will there be any problems I haven't anticipated, such as the two configs conflicting? For example, I notice that there is only one /etc/amandates file that is presumably shared by both configurations. I don't know whether the date to begin incrementals from is from this file or from /var/lib/amanda/gnutar-lists. No, it isn't a good idea, because there's a better solution that doesn't involve this sort of gymnastics: RAIT. You can set up a RAIT between the two tape devices, and then even if a drive is down, your recoveries will work fine. Dustin The configuration I describe solves some problems that RAIT won't. See if there's any error in my reasoning here. The two configurations run on alternating days. One configuration runs more often (3 days/week) on a smaller disk, so these backups only go back a month. The other configuration runs less often (2 days/week) on a larger disk, so these backups go back two months. With RAIT, you get one configuration that stores identically replicated data, so there's no opportunity to tweak one copy to go back further in time (is this true?). Another relevant point is, the second disk was added for extra capacity. With this extra capacity, we can run backups every single day instead of only every two days. Thus, there is more granularity in the backups with this configuration. If a user deletes a file accidentally today, then we are guaranteed to have a copy less than 24 hours old. For backups between 1 and 2 months ago, we don't care as much about the granularity. Whether or not the effort from this config's required gymnastics is offset by these extra advantages, are there any other problems I can anticipate? I'm no Amanda expert, so I fear not hearing the loud alarms that would be set off in a more experienced user's mind when he hears about this odd configuration. Thanks for the quick response, Dustin. You're right that I may be a bit over my head here. :) John
Re: Overlapping backups: should I expect problems?
John Morris wrote: Dustin J. Mitchell wrote: On Mon, Sep 15, 2008 at 11:36 PM, John Morris [EMAIL PROTECTED] wrote: My question is, is this actually a good idea? Will there be any problems I haven't anticipated, such as the two configs conflicting? For example, I notice that there is only one /etc/amandates file that is presumably shared by both configurations. I don't know whether the date to begin incrementals from is from this file or from /var/lib/amanda/gnutar-lists. No, it isn't a good idea, because there's a better solution that doesn't involve this sort of gymnastics: RAIT. You can set up a RAIT between the two tape devices, and then even if a drive is down, your recoveries will work fine. Dustin The configuration I describe solves some problems that RAIT won't. See if there's any error in my reasoning here. The two configurations run on alternating days. One configuration runs more often (3 days/week) on a smaller disk, so these backups only go back a month. The other configuration runs less often (2 days/week) on a larger disk, so these backups go back two months. With RAIT, you get one configuration that stores identically replicated data, so there's no opportunity to tweak one copy to go back further in time (is this true?). Another relevant point is, the second disk was added for extra capacity. With this extra capacity, we can run backups every single day instead of only every two days. Thus, there is more granularity in the backups with this configuration. If a user deletes a file accidentally today, then we are guaranteed to have a copy less than 24 hours old. For backups between 1 and 2 months ago, we don't care as much about the granularity. Whether or not the effort from this config's required gymnastics is offset by these extra advantages, are there any other problems I can anticipate? I'm no Amanda expert, so I fear not hearing the loud alarms that would be set off in a more experienced user's mind when he hears about this odd configuration. Just a couple of comments. I've had a couple of occasions in the last few weeks where I've needed to recover people's mail on the server, because they had errors in how they configured their desktops and lost historical messages that were important to them. I was able to pinpoint the day it was sure to be on the server and pull it back. In one case I had to go back to 3 different dates including an archive from June. Having each and every day on backups proved important. Having to figure out not just the date, but then which configuration to recover from, based on which day of the week that date was, complicates it some. But also, if you lose one of your drives (the reason you've configured it this way), you also actually lose some of your dates. That's an actual loss of data. With RAIT, if you lose a drive, you haven't lost any data. Also, with RAIT, the actually backup and replication of the data is more efficient. You are only accessing the clients to back up the data once and getting the RAIT copying without an additional transfer across the network. So your data redundancy is both more complete and more efficient. If your drive capacities are different based on purchasing history, and RAIT seems not quite right because of that, I would just make sure I have suitable drives for the design. Drives are relatively inexpensive these days. -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst [EMAIL PROTECTED] --- Erdös 4
Re: Overlapping backups: should I expect problems?
On Tue, Sep 16, 2008 at 04:41:04PM +0800, John Morris wrote: Dustin J. Mitchell wrote: On Mon, Sep 15, 2008 at 11:36 PM, John Morris [EMAIL PROTECTED] wrote: My question is, is this actually a good idea? Will there be any problems I haven't anticipated, such as the two configs conflicting? For example, I notice that there is only one /etc/amandates file that is presumably shared by both configurations. I don't know whether the date to begin incrementals from is from this file or from /var/lib/amanda/gnutar-lists. No, it isn't a good idea, because there's a better solution that doesn't involve this sort of gymnastics: RAIT. You can set up a RAIT between the two tape devices, and then even if a drive is down, your recoveries will work fine. Dustin The configuration I describe solves some problems that RAIT won't. See if there's any error in my reasoning here. I'll add another yes to Dustin's and Chris' suggestion of RAIT. The two configurations run on alternating days. One configuration runs more often (3 days/week) on a smaller disk, so these backups only go back a month. The other configuration runs less often (2 days/week) on a larger disk, so these backups go back two months. With RAIT, you get one configuration that stores identically replicated data, so there's no opportunity to tweak one copy to go back further in time (is this true?). This is not the same thing but may have some benefits. If your average daily backup size is smaller, you would be able to retain more daily backups in the same disk space. Typically that would be done by lengthening the dumpcycle. Maybe that is not appropriate for all DLE's you define, but keep in mind that dumpcycle can now be defined for individual dumptypes. So if you can lengthen the dumpcycle for some basically static DLEs, maybe you can increase the total number of dumps being retained. Two examples of where I've used this technique are a file system I used to contain CD and DVD .iso files and another containing only copies of equipment user guides and other reference material. Neither ever changes unless I added something. So month long dumpcycles were not a problem. Sometimes when I've added a cd/dvd image and it kept getting backed up by the incrementals, I forced an early level 0 on that DLE. I mean how many backup copies of fedora.iso do I need? ;) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 12027 Creekbend Drive (703) 787-0884 Reston, VA 20194 (703) 787-0922 (fax)
Overlapping backups: should I expect problems?
Hi, We're doing our on-site backups using virtual tapes. Two hard drives are dedicated to this. In an attempt to be clever, I set up two configurations, DS2 and DS3, that back up the same filesystems, but whose virtual tapes are on separate disks. The idea is that if we need to restore, and one of the disks turns out to have a problem, that we'll still have the other disk with a complete level 0 and incrementals. The configurations run on alternating days. Does this make sense? My question is, is this actually a good idea? Will there be any problems I haven't anticipated, such as the two configs conflicting? For example, I notice that there is only one /etc/amandates file that is presumably shared by both configurations. I don't know whether the date to begin incrementals from is from this file or from /var/lib/amanda/gnutar-lists. Thanks for any hints! This configuration has been running for a few weeks without any evident problems so far, except for one DLE whose incrementals are always the same size as the level 0s, despite that there are very few changes to the filesystem; I'm still working that one out. John
Re: Overlapping backups: should I expect problems?
On Mon, Sep 15, 2008 at 11:36 PM, John Morris [EMAIL PROTECTED] wrote: My question is, is this actually a good idea? Will there be any problems I haven't anticipated, such as the two configs conflicting? For example, I notice that there is only one /etc/amandates file that is presumably shared by both configurations. I don't know whether the date to begin incrementals from is from this file or from /var/lib/amanda/gnutar-lists. No, it isn't a good idea, because there's a better solution that doesn't involve this sort of gymnastics: RAIT. You can set up a RAIT between the two tape devices, and then even if a drive is down, your recoveries will work fine. Dustin -- Storage Software Engineer http://www.zmanda.com
