Re: Backup advice
On Wed, May 23, 2007 at 07:27:05PM -0400, Jason Lixfeld wrote: > So I feel a need to start backing up my servers. To that end, I've decided > that it's easier for me to grab an external USB drive instead of a tape. It Buy at least two, and keep one off-site. > would seem dump/restore are the tools of choice. My backup strategy is > pretty much "I don't want to be screwed if my RAID goes away". That said I > have a few questions along those lines: > > - Most articles I've read suggest a full backup, followed by incremental > backups. Is there any real reason to adopt that format for a backup > strategy like mine, or is it reasonable to just do a dump 0 nightly? I > think the only reason to do just one full backup per 'cycle' would be to > preserve system resources, as I'm sure it's fairly taxing on the system > during dump 0 times. Depending on the size of your data, a level 0 dump could take a couple of hours. Unless you have a terabyte raid array, in which case a single USB disk probably won't cut it. :) On the other hand, if your dataset changes rapidly you might not save much with incremental dumps. You can save time by setting the nodump flag on directories that contain files that you don't really nead or can easily replace, such as /usr/obj, /usr/ports/distfiles, /tmp et cetera. > - Can dump incrementally update an existing dump, or is the idea that a dump > is a closed file and nothing except restore should ever touch it? You cannot update a dump file, AFAIK. > - How much does running a backup through gzip actually save? Is taxing the > system to compress the dump and the extra time it takes actually worth it, > assuming I have enough space on my backup drive to support a dump 0 or two? It depends. On a normal filesystem you save about 50% with gzip. But if you have lots of (already compressed) audio and picture data there are almost no savings. Compressing with gzip shouldn't tax the system too much, unless it's very old. Using bzip2 usually isn't worth it. It takes much longer and maxes out the CPU on my 2,4 GHz athlon64. Do not forget the -L flag if you're dumping a live filesystem! > - Other folks dumping to a hard drive at night? Care to share any of your > experiences/rationale? My desktop machine's file systems are backed up every week to a USB drive, using gzipped dumps. Every month I start with a new level 0 dump. When I run out of space I delete the oldest set of dumps. When I nuked my /usr parition by accident I was very happy to be able to restore things with the tools in /rescue, without first having to rebuild a lot of ports. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpOzVOMwrG8f.pgp Description: PGP signature
Re: Backup advice
On Thu, May 24, 2007 at 03:20:28AM -0400, Jason Lixfeld wrote: > > On 24-May-07, at 3:16 AM, Olivier Nicole wrote: > > >>2 x system space would be enough for a full dump plus plenty of > >>increments, I'd say. No? Is there a rule of thumb? 3x? 4x? > > > >That depends how much your file system change. If every ficle change > >befor the incremental run, dump 1 will be equal to dump 2, and 2x will > >be enough for just dump0 and dump 1. > > > >There is no rule. > > How would one go about gauging their system for the number of file > system changes to determine a suitable amount of backup space? To some extent, you must know how you use the system. After that, it is just a matter of experience with that system. After you have done this dump cycle a few times you will begin to see a pattern. jerry > > >Olivier > > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On Thu, May 24, 2007 at 03:10:43AM -0400, Jason Lixfeld wrote: > > On 24-May-07, at 12:33 AM, Doug Hardie wrote: > > > > >On May 23, 2007, at 19:03, Jason Lixfeld wrote: > > > >> > >>On 23-May-07, at 9:23 PM, Doug Hardie wrote: > >> > >>>The criteria for selecting a backup approach is not the backup > >>>methodology but the restore methodology. > >> > >>Excellent point. > >> > >>Perhaps I'm asking the wrong question, so let me try it this way > >>instead: > >> > >>software goes. > > > >What kind of data are you backing up? If you are backing up the > >system and your data then you have to be very careful about links. > >Some backup solutions will copy the files as separate files. When > >you restore the link is gone. An update to one of the linked files > >will no longer be seen by the other names. The OS uses a lot of > >links. If all you are backing up is data, its probably not an > >issue. Yes, I neglected to mention the issue of veracity of the backups. dump/restore is the only one that completely handles the hard links the way you want. It may also be the only one that handles ACLs properly if you use those. I haven't examined that issue. > > Dump seems to be the best at doing what I'm looking to do. Better > than tar or rsync. I think dd would beat out dump, but dd is far > less of a backup tool than dump is, so I think dump is still the > winner. The caveat of a full dump taking the most time and resources > can be reasonably mitigated by doing a full dump every X intervals > and incremental in between. It seems to be a fair compromise seeing > as how cheap hard drive space is these days. Note that dd is not really a backup utility. It is a data copy utility. If you have a catastrophic failure on a disk and need to replace it, there is every likelihood that the new drive will NOT be exactly like the old one. Doing a disk build with fdisk, bsdlabel and newfs and restoring from dumps would get you exactly what you want. But, using dd would not. You would have an exact copy of the old boot sectors, MBR, partition tables which would not be correct for the new drive (although they might work, sort of). > > 2 x system space would be enough for a full dump plus plenty of > increments, I'd say. No? Is there a rule of thumb? 3x? 4x? Depends a lot on how much your data changes. In your case, that would include log files, since you intend to back up the whole system.Other than log files, the system itself will not change a lot. But, I have no idea of what your data does. I would feel comfortable with 2X for my sort of stuff and be able to do a full, plus maybe half a dozen incrementals or so. But even 4X might not cover it for some volatile systems. > As far as restoring goes, let's assume my machine blew up one full > backup and 15 increments ago and I want to restore the entire system > in it's entirety from my backup. How is that done? Point restore to > the last incremental and it figures it out for itself, or is it a > manual process where I have to figure out what backups consist of the > complete system? No, you first restore the full dump and continue through the incrementals in order of increasing level. If you made more than one incremental at a specific level, then only restore from the last one made. > > >One backup disk is not all that great a safety approach. You will > >never know if that drive has failed till you try and use it. Then > >its too late. Failures do not require that the drive hardware has > >failed. Any interruption in the copy can cause an issue that may > >not be detected during the backup. Sectors generally don't just go > >bad sitting on the shelf, but it does happen. That was a > >significant problem with tapes. Generally 25% of the tapes I used > >to get back from off-site storage after a month were no longer > >readable. > > There has to be some way for the OS to know if a drive is bad, or to > verify the state of the data that was just copied from one location > to another. Is there no method of doing error correction? My laptop > backup programs I've been using for years shows me information at the > end of the run: Files copied, Speed, Time, Errors, etc. The OS does see read/write errors on a disk and reports them. dump will tell you if it thinks there was a media error, but that doesn't tell you much - and probably doesn't really on your laptop. It is probably a false sense of security. There used to be a verify option on dump, or maybe it was in some other proprietary version of UNIX I worked on. But it made dumps take so long that we quickly gave up using it. It required reading back the media and comparing it to the original. Then the verify often failed because a file was changed or deleted between the time it was written and the time it was verified. So, the verify was not useful. > > If a UNIX backup process is as unreli
Re: Backup advice
On Wed, May 23, 2007 at 10:03:40PM -0400, Jason Lixfeld wrote: > > On 23-May-07, at 9:23 PM, Doug Hardie wrote: > > >The criteria for selecting a backup approach is not the backup > >methodology but the restore methodology. > > Excellent point. > > Perhaps I'm asking the wrong question, so let me try it this way > instead: > > I'm looking for a backup solution that I can rely on in the event I > have a catastrophic server failure. Ideally this backup would look > and act much like a clone of the production system. In the worse > case, I'd re-format the server array and copy the clone back to the > server, setup the boot blocks, and that would be it. > > Ideally this clone should be verifiable, meaning I should be able to > verify it's integrity so that it's not going to let me down if I need > it. > > I'm thinking external USB hard drive of at least equal size to the > server array size as far as hardware goes, but I'm lost as far as > software goes. Sounds like you are not quite as critical as the other post - somewhere in between. If you want an immediately available clone, then the best thing is to have an identical machine, preferably off-site and maintain it as a clone, probably using rsync, although you can reasonably use dump/restore for that too. If you need calls for just being back up in a reasonable length of time then you might prefer dumping to some media and if the need comes to restore, then you would have to recreate the disk structure - using fdisk, bsdlabel and newfs from the fixit image on the install CD or use sysinstall to run them for you. (I suggest that any serious System Manager become familiar with fdisk, bsdlabel and newfs, even if you usually let sysinstall handle them for you) Then you would use restore to pull the dumps back in. If your system is super critical as Doug Hardie posted about his, then you may want to use some combination of rsync-ing to a close and making dumps and consider storing some of these off-site. jerry > > Any advice appreciated. > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On Wed, May 23, 2007 at 07:27:05PM -0400, Jason Lixfeld wrote: > So I feel a need to start backing up my servers. To that end, I've > decided that it's easier for me to grab an external USB drive instead > of a tape. It would seem dump/restore are the tools of choice. My > backup strategy is pretty much "I don't want to be screwed if my RAID > goes away". That said I have a few questions along those lines: A popular sentiment. > - Most articles I've read suggest a full backup, followed by > incremental backups. Is there any real reason to adopt that format > for a backup strategy like mine, or is it reasonable to just do a > dump 0 nightly? I think the only reason to do just one full backup > per 'cycle' would be to preserve system resources, as I'm sure it's > fairly taxing on the system during dump 0 times. Yes, dump/restore is generally the way to go, unless you have not set up your partitions conveniently to separate what you want to dump from what you do not want to dump. The main reason to do a full dump followed by a series of incrementals is to save resources. This includes dump time as well as media to receive the dump[s]. If you happen to be using tape for example, a large full dump may take several tapes for each dump, but an incremental may then take only one for each. There is one more thing to consider. The way dump works is that it starts by making a large list of all the stuff it will dump. Then it starts writing to media (tape, disk file, network, whatever). On systems where files change frequently, especially new ones being added and old ones being deleted, it is quite possible, even probable that there will be changes between the time the index list is made and when the dump of a particular file/directory is written. dump and restore handle this with now problem and just a little warning message, but it makes the backup a little less meaningful. You will often see messages from restore saying it is skipping a file it cannot find. That is because the file was deleted from disk after the list was made, but before the data was written to media. Files created after the list was made will not be dumped until the next time dump is run. Files that are modified after the list was made will only be dumped if they were also modified before the list was made. That said, if the amount I am backing up takes less than about an hour for a level 0 and I have room for it, I always do the full dump each time and ignore the incremental issue.In cases where the full dump takes a long time, but there are typically not a lot of changes on the system, I usually do a level 0, followed only by a series of level 1 dumps until they tend to get large and then start another level 0 dump. > - Can dump incrementally update an existing dump, or is the idea that > a dump is a closed file and nothing except restore should ever touch it? No, dump does not work that way. It works on complete files. It keeps a record of when the most recent dumps were done along with the level of the dump that was done - in a file called /etc/dumpdates. Then, when it makes its list of files and directories to dump, it looks at the date the file was changed. If the change was more recent than the next lower dump level than currently being done, it adds the file to the list and dumps it to the incremental media. Full dumps just set the date of most recent dump to the "epoch" (1970) so any file or directory changed since then is dumped. Since that is the nominal beginning of time for UNIX of any time, all files will be changed since then and thus be added to the list to be dumped. So, essentially, yes to the second part of the question. A dump file might as well be considered a closed file. Incrementals are additional closed files. > > - How much does running a backup through gzip actually save? Is > taxing the system to compress the dump and the extra time it takes > actually worth it, assuming I have enough space on my backup drive to > support a dump 0 or two? As with other data, it depends on the data. I never compress dumps. Maybe I am a little supersticious, but I don't want any other complication potentially in the way under the circumstance when I find I need something from the dump.Also, you would have to uncompress the dump before you could do an 'interactive' restore or any other partial restore. jerry > > - Other folks dumping to a hard drive at night? Care to share any of > your experiences/rationale? > > Thanks in advance. > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]
Re: Backup advice
On 24-May-07, at 3:43 AM, Doug Hardie wrote: Rsync will leave you with a duplicate of the drive. You could pretty much boot off it and run. You would need to configure the drive and install a boot loader though. The boot off and run is more in-line with what I want to do, so I will go the rsync route instead of the dump/restore route. Thanks for your feedback, Doug. It's been a great help. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On 24-May-07, at 3:16 AM, Olivier Nicole wrote: 2 x system space would be enough for a full dump plus plenty of increments, I'd say. No? Is there a rule of thumb? 3x? 4x? That depends how much your file system change. If every ficle change befor the incremental run, dump 1 will be equal to dump 2, and 2x will be enough for just dump0 and dump 1. There is no rule. How would one go about gauging their system for the number of file system changes to determine a suitable amount of backup space? Olivier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
> 2 x system space would be enough for a full dump plus plenty of > increments, I'd say. No? Is there a rule of thumb? 3x? 4x? That depends how much your file system change. If every ficle change befor the incremental run, dump 1 will be equal to dump 2, and 2x will be enough for just dump0 and dump 1. There is no rule. Olivier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On 24-May-07, at 12:33 AM, Doug Hardie wrote: On May 23, 2007, at 19:03, Jason Lixfeld wrote: On 23-May-07, at 9:23 PM, Doug Hardie wrote: The criteria for selecting a backup approach is not the backup methodology but the restore methodology. Excellent point. Perhaps I'm asking the wrong question, so let me try it this way instead: I'm looking for a backup solution that I can rely on in the event I have a catastrophic server failure. Ideally this backup would look and act much like a clone of the production system. In the worse case, I'd re-format the server array and copy the clone back to the server, setup the boot blocks, and that would be it. Ideally this clone should be verifiable, meaning I should be able to verify it's integrity so that it's not going to let me down if I need it. I'm thinking external USB hard drive of at least equal size to the server array size as far as hardware goes, but I'm lost as far as software goes. What kind of data are you backing up? If you are backing up the system and your data then you have to be very careful about links. Some backup solutions will copy the files as separate files. When you restore the link is gone. An update to one of the linked files will no longer be seen by the other names. The OS uses a lot of links. If all you are backing up is data, its probably not an issue. I have used both dump and tar successfully. I currently use tar as I have many directories I don't want to backup. Tar requires some care and feeding to handle links properly. It doesn't do it by default. Dump does handle them properly by default. Another option is rsync. The advantage it has is that it only copies the changes in the file. It will run a lot faster than dump or tar which will copy everything each time. You do have to be careful with /dev if you are copying the root partition. I'm backing up my entire system. To me, it's easier this way in the long run. In the event of a failure, you just copy everything from the backup back to the system without the need to worry about re- installing applications, library dependencies, configuration files, nuances, idiosyncrasies, etc. I've been doing this for years with my OS X laptop and it's the quickest way to get back on your feet in a worst case scenario. Dump seems to be the best at doing what I'm looking to do. Better than tar or rsync. I think dd would beat out dump, but dd is far less of a backup tool than dump is, so I think dump is still the winner. The caveat of a full dump taking the most time and resources can be reasonably mitigated by doing a full dump every X intervals and incremental in between. It seems to be a fair compromise seeing as how cheap hard drive space is these days. 2 x system space would be enough for a full dump plus plenty of increments, I'd say. No? Is there a rule of thumb? 3x? 4x? As far as restoring goes, let's assume my machine blew up one full backup and 15 increments ago and I want to restore the entire system in it's entirety from my backup. How is that done? Point restore to the last incremental and it figures it out for itself, or is it a manual process where I have to figure out what backups consist of the complete system? One backup disk is not all that great a safety approach. You will never know if that drive has failed till you try and use it. Then its too late. Failures do not require that the drive hardware has failed. Any interruption in the copy can cause an issue that may not be detected during the backup. Sectors generally don't just go bad sitting on the shelf, but it does happen. That was a significant problem with tapes. Generally 25% of the tapes I used to get back from off-site storage after a month were no longer readable. There has to be some way for the OS to know if a drive is bad, or to verify the state of the data that was just copied from one location to another. Is there no method of doing error correction? My laptop backup programs I've been using for years shows me information at the end of the run: Files copied, Speed, Time, Errors, etc. If a UNIX backup process is as unreliable as you're making it out to be, then I could buy 10 drives and still potentially have each one fail and be screwed if I were to need to rely on it at some point. I'd feel more comfortable backing up off a RAID1 to a single backup drive that provided some sort of error protection/correction/ notification than backing up off a RAID1 to 100 backup drives that didn't give me any indication as to the success of the copy. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On May 23, 2007, at 19:03, Jason Lixfeld wrote: On 23-May-07, at 9:23 PM, Doug Hardie wrote: The criteria for selecting a backup approach is not the backup methodology but the restore methodology. Excellent point. Perhaps I'm asking the wrong question, so let me try it this way instead: I'm looking for a backup solution that I can rely on in the event I have a catastrophic server failure. Ideally this backup would look and act much like a clone of the production system. In the worse case, I'd re-format the server array and copy the clone back to the server, setup the boot blocks, and that would be it. Ideally this clone should be verifiable, meaning I should be able to verify it's integrity so that it's not going to let me down if I need it. I'm thinking external USB hard drive of at least equal size to the server array size as far as hardware goes, but I'm lost as far as software goes. What kind of data are you backing up? If you are backing up the system and your data then you have to be very careful about links. Some backup solutions will copy the files as separate files. When you restore the link is gone. An update to one of the linked files will no longer be seen by the other names. The OS uses a lot of links. If all you are backing up is data, its probably not an issue. I have used both dump and tar successfully. I currently use tar as I have many directories I don't want to backup. Tar requires some care and feeding to handle links properly. It doesn't do it by default. Dump does handle them properly by default. Another option is rsync. The advantage it has is that it only copies the changes in the file. It will run a lot faster than dump or tar which will copy everything each time. You do have to be careful with /dev if you are copying the root partition. One backup disk is not all that great a safety approach. You will never know if that drive has failed till you try and use it. Then its too late. Failures do not require that the drive hardware has failed. Any interruption in the copy can cause an issue that may not be detected during the backup. Sectors generally don't just go bad sitting on the shelf, but it does happen. That was a significant problem with tapes. Generally 25% of the tapes I used to get back from off-site storage after a month were no longer readable. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
Hi, > So I feel a need to start backing up my servers. To that end, I've > decided that it's easier for me to grab an external USB drive instead > of a tape. It's certainly faster, easier and cheaper now days. > It would seem dump/restore are the tools of choice. This really repends of what you are backuping. Dump is OS specific. Ifwhat really, really matters is the data, notthe OS, not the software, tar may be more generic and could even be recovered on a Windows machine. > My > backup strategy is pretty much "I don't want to be screwed if my RAID > goes away". That said I have a few questions along those lines: > - Most articles I've read suggest a full backup, followed by > incremental backups. Is there any real reason to adopt that format > for a backup strategy like mine, or is it reasonable to just do a > dump 0 nightly? I think the only reason to do just one full backup > per 'cycle' would be to preserve system resources, as I'm sure it's > fairly taxing on the system during dump 0 times. Depends on your retore needs. Once you have backup you may soon find out that you want to restore data from 2 days ago, if you keep only the latest dump 0, you cannot. If you plan to keep several runs of dump 0, going incremental will save you time and disk space (at the cost of some time in case of recovery, but we expect recovery will never be needed...) > - Can dump incrementally update an existing dump, or is the idea that > a dump is a closed file and nothing except restore should ever touch it? No. Incremental dumps are just other files, so for 50 GB of data and 5 GB changing each day, you would need a backup space equal to 50GB for dump 0 plus 5GB for each incremental. > - How much does running a backup through gzip actually save? Is > taxing the system to compress the dump and the extra time it takes > actually worth it, assuming I have enough space on my backup drive to > support a dump 0 or two? Not sure if dump is not already compressed. > - Other folks dumping to a hard drive at night? Care to share any of > your experiences/rationale? I have been running Amanda for years, on tape because the tape drive is still solid, next support will be external drive (USB, eSATA, whatever comes next). I backup FreeBSD, some Linuxes, various windows... I use Gnu tar with Amanda, so my tapes can be read on many OS'es. Olivier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On Wednesday 23 May 2007 21:03:40 Jason Lixfeld wrote: > On 23-May-07, at 9:23 PM, Doug Hardie wrote: > > The criteria for selecting a backup approach is not the backup > > methodology but the restore methodology. > > Excellent point. > > Perhaps I'm asking the wrong question, so let me try it this way > instead: > > I'm looking for a backup solution that I can rely on in the event I > have a catastrophic server failure. Ideally this backup would look > and act much like a clone of the production system. In the worse > case, I'd re-format the server array and copy the clone back to the > server, setup the boot blocks, and that would be it. > > Ideally this clone should be verifiable, meaning I should be able to > verify it's integrity so that it's not going to let me down if I need > it. > > I'm thinking external USB hard drive of at least equal size to the > server array size as far as hardware goes, but I'm lost as far as > software goes. > > Any advice appreciated. > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "[EMAIL PROTECTED]" in our enterprise, we use Veritas NetBackup. expensive, but has been 100% reliable for us for years. its unix agent is currently being used on everything from linux, to osx and freebsd. unfortunatly, this one doesnt fall under the "free of cost" category, but veritas tech support that comes with it has always been top notch. just thought id throw that out there, incase this is for work and there is a budget line item to take care of this project :) -- Jonathan Horne http://dfwlpiki.dfwlp.org [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On 23-May-07, at 9:23 PM, Doug Hardie wrote: The criteria for selecting a backup approach is not the backup methodology but the restore methodology. Excellent point. Perhaps I'm asking the wrong question, so let me try it this way instead: I'm looking for a backup solution that I can rely on in the event I have a catastrophic server failure. Ideally this backup would look and act much like a clone of the production system. In the worse case, I'd re-format the server array and copy the clone back to the server, setup the boot blocks, and that would be it. Ideally this clone should be verifiable, meaning I should be able to verify it's integrity so that it's not going to let me down if I need it. I'm thinking external USB hard drive of at least equal size to the server array size as far as hardware goes, but I'm lost as far as software goes. Any advice appreciated. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On May 23, 2007, at 16:27, Jason Lixfeld wrote: So I feel a need to start backing up my servers. To that end, I've decided that it's easier for me to grab an external USB drive instead of a tape. It would seem dump/restore are the tools of choice. My backup strategy is pretty much "I don't want to be screwed if my RAID goes away". That said I have a few questions along those lines: - Most articles I've read suggest a full backup, followed by incremental backups. Is there any real reason to adopt that format for a backup strategy like mine, or is it reasonable to just do a dump 0 nightly? I think the only reason to do just one full backup per 'cycle' would be to preserve system resources, as I'm sure it's fairly taxing on the system during dump 0 times. - Can dump incrementally update an existing dump, or is the idea that a dump is a closed file and nothing except restore should ever touch it? - How much does running a backup through gzip actually save? Is taxing the system to compress the dump and the extra time it takes actually worth it, assuming I have enough space on my backup drive to support a dump 0 or two? - Other folks dumping to a hard drive at night? Care to share any of your experiences/rationale? The criteria for selecting a backup approach is not the backup methodology but the restore methodology. What failures can you tollerate and what can you not afford to lose forever. Backup to a single disk leaves you with a big vulnerability if something is wrong with that backup. You stand to lose pretty much everything. If everything is stored in one location, what happens if it vanishes? My approach is dictated by the restore requirements. We have some databases that are absolutely critical. Loss of those is the end of the world. Every module that updates the database also writes a copy of the updated transaction to a log file. I rsync the log file to multiple machines separated by many miles every 10 minutes. The complete database is dumped everynight and that is also rsync'd to the same machines daily. Each of the backup machines retains several months of the full dumps and the transaction logs. From that presuming that one site remains available, I can reconstruct all but the last 10 minutes of the database. The complete system is dumped to a disk on one of the local servers weekly. A DVD is cut from that and taken off-site for retention. Actually all that is needed is the local software source and the config files as FreeBSD is easily replaced. However, since there are always times were some of the ports may not be the latest version it is easier to have the actual ones in use rather than having to checkout newer versions. The full dump is also rsync'd weekly to a couple of off-site machines. Whatever you decide to do, figure out how to recover and test it. Finding out you need something you didn't save is much less traumatic if you find out before a failure occurs. I test my restore procedures yearly. I have two machines I use at home for doing test recoveries. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
On 24/05/07, Howard Goldstein <[EMAIL PROTECTED]> wrote: Jason Lixfeld wrote: > - Other folks dumping to a hard drive at night? Care to share any of > your experiences/rationale? Not with dump/restore. After using amanda and a tape drive for eons I'm now happy with a bacula solution to backup 2 freebsds and a windows machine. It does incrementals except on Saturday night when it alternates between a differential (sort of a mass incremental from the last full) and a full backup to a cheap IDE drive. Every Sunday I copy the IDE drive to a USB drive and take it offsite and bring back another one. After restoring from scratch - power supply frying the entire RAID array on my desktop -STABLE machine - I think the advantages of dump are certainly there but for my apps, where I don't have any huge sparse files or a lot of hard links other than whatever gets installed with a fresh install (if anything) to worry about - they're outweighed by the convenience of bacula where I can go back to a point in time. YMMV... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" We have something similar, we use rsnapshot (/usr/ports/sysutils/rsnapshot/) http://www.rsnapshot.org/ The first rsnapshot takes up the full amount of space and rsnapshot there after only takes up the space of those files that have changed. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup advice
Jason Lixfeld wrote: - Other folks dumping to a hard drive at night? Care to share any of your experiences/rationale? Not with dump/restore. After using amanda and a tape drive for eons I'm now happy with a bacula solution to backup 2 freebsds and a windows machine. It does incrementals except on Saturday night when it alternates between a differential (sort of a mass incremental from the last full) and a full backup to a cheap IDE drive. Every Sunday I copy the IDE drive to a USB drive and take it offsite and bring back another one. After restoring from scratch - power supply frying the entire RAID array on my desktop -STABLE machine - I think the advantages of dump are certainly there but for my apps, where I don't have any huge sparse files or a lot of hard links other than whatever gets installed with a fresh install (if anything) to worry about - they're outweighed by the convenience of bacula where I can go back to a point in time. YMMV... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
