Re: Backup with dd?
Garance A Drosihn wrote: This assumes you're running 5.3-stable or 6.x-current. I am not sure how well snapshots would work on 4.x-stable. Not at all, unfortunately. snapshots are a UFS2 feature only available in 5.x or above. However, sys-admins have been making backups without the benefit of snapshots for decades: usually if you can arrange for anything that uses the partition to be quiescent or turned off while the backup is being run, you should succeed. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 8 Dane Court Manor School Rd PGP: http://www.infracaninophile.co.uk/pgpkey Tilmanstone Tel: +44 1304 617253 Kent, CT14 0JL UK signature.asc Description: OpenPGP digital signature
Re: Backup with dd?
On Tue, 4 Jan 2005 04:27 am, Eric F Crist wrote: > Hello all, > > I've decided to try doing a complete system backup, attempting a > bit-for-bit copy. A friend told me to try the following: > > # dd if=/dev/ad4 of=/dev/ad6 > > Both drives are identical SATA150. Is this the best way? I'm hope to > be able to do a daily/weekly backup this way, and if my primary drive > fails, switch the cables and just reboot. > There can be a few major problems in using dd for backup: If the source is mounted then it can change during backup and lead to a bckup that is not self consistent. At very least when you attempt to boot the backup you will find it is not marked clean and fsck will spend some time fixing it. This problem can probably be avoided by having the source (and destination) unmounted during the backup; perhaps by running the backup from a "fixit" system. The other problem is that it normally will take a very long time. I've tried this in the past -- can't remember details of disk sizes etc. but do recall times in excess of 20 hours, during which the system should be otherwise as dormant as possible. Identical drives are like identical twins; ie they are not quite identical. While it is not likely to be a problem it could be. Safer to use equal sized slices on each drive rather than complete drive. My preference would be to use dump and restore on a partition by partition basis; which is more or less the intended application of these utilities. Malcolm ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup with dd?
Eric F Crist <[EMAIL PROTECTED]> writes: > I've decided to try doing a complete system backup, attempting a > bit-for-bit copy. A friend told me to try the following: > > # dd if=/dev/ad4 of=/dev/ad6 > > Both drives are identical SATA150. Is this the best way? Not particularly. The primary advantage of using dd for backups is that unlike other methods, it makes a faithful copy of the unused bits on the disk. Personally, I've never found this useful, but if you need it, you need it. >I'm hope to > be able to do a daily/weekly backup this way, and if my primary drive > fails, switch the cables and just reboot. dump/restore would be the obvious way to do that. Note, however, that it doesn't help much if your machine is hacked (because you likely won't notice right away, and the "backup" will contain subverted executables and/or devices just like the running disk). Nor will it protect you from accidental deletion of important files or loss of the computer in a fire. You'll still need traditional backups for that. Pretty much the only thing from which the "dd" method protects you is downtime due to losing a disk, and RAID is more likely to do that successfully than copying the disk bit-by-bit. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup with dd?
At 1:03 PM -0600 1/3/05, Eric F Crist wrote: On Jan 3, 2005, at 12:46 PM, Andrew P. wrote: Eric F Crist wrote: You seem to be under the impression that I'm doing this for the sole reason of a disk crash. I'm actually doing it for more than just that reason. For example, if my system gets hacked, most hackers will probably not care about an unmounted hard drive, and screw with the current mounted partitions. [...etc...] Backing up with dd is ultimately straightforward, but is not a good idea at all. The matter is when dd is running, the source may be modified and the copy might be inconsistent. Software RAID should be the best option for your task: you can mirror a drive to a second one and then just plug the second one out of your computer. Best wishes, Andrew P. Is this vinum? Fairly difficult to setup, or is it straight-forward? Before I delve into that, any setup recommendations? Software raid seems like a messy way to handle this, when all he wants is straight copies of the partitions. Much easier to mount and umount some destination-partition(s), than it is to unplug a hard drive. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup with dd?
At 11:57 AM -0600 1/3/05, Eric F Crist wrote: Hello all, I've decided to try doing a complete system backup, attempting a bit-for-bit copy. A friend told me to try the following: # dd if=/dev/ad4 of=/dev/ad6 Both drives are identical SATA150. Is this the best way? While that will probably work, it is also somewhat risky to make a direct copy of a disk that you are actively using. You can end up with a copy that has inconsistencies, because of changes that happen on the source disk during the time it takes to do a copy. And if you are copying a huge disk, it *will* take a significant amount of time to perform that copy. By "inconsistent", I mean that when you boot up on the copy, the initial 'fsck' will fail because of inconsistencies on the disk. I have done 'dd' copies like this. I have seen fsck failures... I'm hope to be able to do a daily/weekly backup this way, and if my primary drive fails, switch the cables and just reboot. You would be better to do the copies on a per-partition basis, and first create a UFS snapshot of each partition, and then use the snapshot as the source for your copy. I actually use a 'dump -L' command, combined with 'restore'. The -L option causes dump to automatically create the snapshot for the partition you specified. It uses the snapshot for the copy, and then destroys the snapshot when the copy has finished. This assumes you're running 5.3-stable or 6.x-current. I am not sure how well snapshots would work on 4.x-stable. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup with dd?
On Jan 3, 2005, at 12:46 PM, Andrew P. wrote: Eric F Crist wrote: On Jan 3, 2005, at 12:15 PM, Henry Miller wrote: This might work, but it isn't best. I can think of the following objections: You have no protection at all while the copy is in progress. You have overwritten part of the old backup, but not enough to be consistent. You have made no provision for data loss because of anything other than a failing drive. If your house burns down you can't get your data. (not strictly true, you can recover accidently deleted files so long as you do the undelete before the next time you do the backup) FreeBSD has a few different RAID options. With the right setup you can achieve disk reliability, and not have to switch cables on reboot. You seem to be under the impression that I'm doing this for the sole reason of a disk crash. I'm actually doing it for more than just that reason. For example, if my system gets hacked, most hackers will probably not care about an unmounted hard drive, and screw with the current mounted partitions. Also, these drives wouldn't really be at the same point of this hypothetical drive failure, since one hard drive will only be used roughly once a week, while the other is in a constant state of use. Most of my user-data is destined for a RAID-5 array that's roughly 1.2TB, so that's got it's own backup. This is simply for use in an emergency, so I don't HAVE to rebuild. Quite frankly, I don't have time to sit here and rebuild this system again any time soon. This configuration I'm trying is ideal, with minimal interference. I'm going to be installing removable drive bays so that my roommate is able to simple swap drive positions and reboot the system (it's headless, and he's not very tech savvy in this regard). Backing up with dd is ultimately straightforward, but is not a good idea at all. The matter is when dd is running, the source may be modified and the copy might be inconsistent. Software RAID should be the best option for your task: you can mirror a drive to a second one and then just plug the second one out of your computer. Best wishes, Andrew P. Is this vinum? Fairly difficult to setup, or is it straight-forward? Before I delve into that, any setup recommendations? Thanks. ___ Eric F Crist "I am so smart, S.M.R.T!" Secure Computing Networks -Homer J Simpson PGP.sig Description: This is a digitally signed message part
Re: Backup with dd?
Eric F Crist wrote: On Jan 3, 2005, at 12:15 PM, Henry Miller wrote: This might work, but it isn't best. I can think of the following objections: You have no protection at all while the copy is in progress. You have overwritten part of the old backup, but not enough to be consistent. You have made no provision for data loss because of anything other than a failing drive. If your house burns down you can't get your data. (not strictly true, you can recover accidently deleted files so long as you do the undelete before the next time you do the backup) FreeBSD has a few different RAID options. With the right setup you can achieve disk reliability, and not have to switch cables on reboot. You seem to be under the impression that I'm doing this for the sole reason of a disk crash. I'm actually doing it for more than just that reason. For example, if my system gets hacked, most hackers will probably not care about an unmounted hard drive, and screw with the current mounted partitions. Also, these drives wouldn't really be at the same point of this hypothetical drive failure, since one hard drive will only be used roughly once a week, while the other is in a constant state of use. Most of my user-data is destined for a RAID-5 array that's roughly 1.2TB, so that's got it's own backup. This is simply for use in an emergency, so I don't HAVE to rebuild. Quite frankly, I don't have time to sit here and rebuild this system again any time soon. This configuration I'm trying is ideal, with minimal interference. I'm going to be installing removable drive bays so that my roommate is able to simple swap drive positions and reboot the system (it's headless, and he's not very tech savvy in this regard). Backing up with dd is ultimately straightforward, but is not a good idea at all. The matter is when dd is running, the source may be modified and the copy might be inconsistent. Software RAID should be the best option for your task: you can mirror a drive to a second one and then just plug the second one out of your computer. Best wishes, Andrew P. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Backup with dd?
On Jan 3, 2005, at 12:15 PM, Henry Miller wrote: This might work, but it isn't best. I can think of the following objections: First, not all "identical" drives are identical. It isn't uncommon for the factory to give slightly different sector counts for drives of the same model, when something in manufacturing changes. (perhaps a new defective sector algorithm). This would cause the backup to fail on those last sectors of the drive. If the drives are really identical, odds are they have the same types of defects, and they will fail at about the same time. That is when one disk fails, the other might not be far behind! You have no protection at all while the copy is in progress. You have overwritten part of the old backup, but not enough to be consistent. You have made no provision for data loss because of anything other than a failing drive. If your house burns down you can't get your data. (not strictly true, you can recover accidently deleted files so long as you do the undelete before the next time you do the backup) FreeBSD has a few different RAID options. With the right setup you can achieve disk reliability, and not have to switch cables on reboot. (With your setup you don't have to either, just tell the BIOS to boot from the other drive if the first one cannot boot) This is more work to setup, but more people understand it, so if something happens to you someone else is more likely to figure out what is going on. (this may or may not matter to you) With a good RAID setup, FreeBSD can keep operating even after the disk crashes, while your setup requires manual intervention. If you must have 24x7 operation (web server), then you need RAID. If you don't need 24x7, consider crashed disks an excuse to re-setup your system, in my experience by the time your disk crashes you will have a lot of cruft that you are meaning to remove, so this is a good excuse to re-install. Thanks for the reply. First off, please reply to the list, so that these emails can be properly archived. This can be accomplished by using reply-all instead of just reply. You seem to be under the impression that I'm doing this for the sole reason of a disk crash. I'm actually doing it for more than just that reason. For example, if my system gets hacked, most hackers will probably not care about an unmounted hard drive, and screw with the current mounted partitions. Also, these drives wouldn't really be at the same point of this hypothetical drive failure, since one hard drive will only be used roughly once a week, while the other is in a constant state of use. Most of my user-data is destined for a RAID-5 array that's roughly 1.2TB, so that's got it's own backup. This is simply for use in an emergency, so I don't HAVE to rebuild. Quite frankly, I don't have time to sit here and rebuild this system again any time soon. This configuration I'm trying is ideal, with minimal interference. I'm going to be installing removable drive bays so that my roommate is able to simple swap drive positions and reboot the system (it's headless, and he's not very tech savvy in this regard). Thanks again for the reply. ___ Eric F Crist "I am so smart, S.M.R.T!" Secure Computing Networks -Homer J Simpson PGP.sig Description: This is a digitally signed message part
Backup with dd?
Hello all, I've decided to try doing a complete system backup, attempting a bit-for-bit copy. A friend told me to try the following: # dd if=/dev/ad4 of=/dev/ad6 Both drives are identical SATA150. Is this the best way? I'm hope to be able to do a daily/weekly backup this way, and if my primary drive fails, switch the cables and just reboot. TIA ___ Eric F Crist "I am so smart, S.M.R.T!" Secure Computing Networks -Homer J Simpson PGP.sig Description: This is a digitally signed message part
