Re: Backup with mtree and rsync?
schu...@ime.usp.br writes: I have been wondering whether it is possible to create a backup system using mtree and rsync. Essentially, the user would create a mtree specification of the source directory and copy it over to the destination directory with rsync. Any changes in the destination could then be detected before restoring with the mtree specification, which should contain strong hashes of the files and should not contain the nlink keyword. The problem is that mtree would be too slow. It would recompute the hashes of big files even when they did not change from the last backup. Therefore, I would like to ask if there is an easy way to accomplish the following. Let a mtree specification of a directory from a certain point in the past be given. Also, assume that a (regular) file below that directory has not changed if its current modification time (mtime) equals its modification time in the past specification. Produce as output the new mtree specification for the directory without reading these files. This is somewhat like rsync does to perform incremental backups. Except that you have a spec for mtree to be sure the backup copy hasn't been corrupted. I don't see any way to do this directly. What you probably want to do is use find(1) to pick out the new files to check, and then merge the changes into the old mtree(8) spec. Not trivial, but the spec syntax is intended to be easy to parse, so it shouldn't be that hard either. P.S.: As an aside, is there an utility in the base system that can reproduce the behavior of `rsync --delete -a dir0/ dir1/`? It's possible that the mtree support in tar(8) might be able to do it, but it would probably be a lot slower. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup with mtree and rsync?
I don't see any way to do this directly. What you probably want to do is use find(1) to pick out the new files to check, and then merge the changes into the old mtree(8) spec. Not trivial, but the spec syntax is intended to be easy to parse, so it shouldn't be that hard either. What I am currently doing somewhat fits your description. I feed find output into a C program that merges the old description with the directory state to produce a new description. However, I use a format different than mtree. I was seeking a shorter, more elegant, solution. It's possible that the mtree support in tar(8) might be able to do it, but it would probably be a lot slower. Wait, can tar be used to remove files? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup with mtree and rsync?
On Tue, 08 Jan 2013 15:57:39 -0200, schu...@ime.usp.br wrote: It's possible that the mtree support in tar(8) might be able to do it, but it would probably be a lot slower. Wait, can tar be used to remove files? No (not directly, except overwriting directories with content), but cpdup can; see man cpdup for details and inspiration. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup with mtree and rsync?
On Sat, Jan 5, 2013 at 8:12 PM, schu...@ime.usp.br wrote: I have been wondering whether it is possible to create a backup system using mtree and rsync. Essentially, the user would create a mtree specification of the source directory and copy it over to the destination directory with rsync. Any changes in the destination could then be detected before restoring with the mtree specification, which should contain strong hashes of the files and should not contain the nlink keyword. A little bit off-topic, but there is a small tool that does something similar to your suggested `mtree` usage, but specifically tailored for backups, `rdup`: http://miek.nl/projects/rdup Although I've not used it myself (I use `rdiff-backup` and on Linux), the idea is pretty similar with what you want to achieve: * you run `rdup` with an old descriptor file plus a target path, and in turn it generates: * a new descriptor file; * a list of files that should be backed up; * you then decide what you do with the list of files to be backed-up (i.e. put them in a `tar`, `rysnc` them to a server, etc.); Hope it helps, Ciprian. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup with mtree and rsync?
I apparently reinvented the wheel. :-) Thanks for the link, it is indeed very inspiring. Quoting Ciprian Dorin Craciun ciprian.crac...@gmail.com: On Sat, Jan 5, 2013 at 8:12 PM, schu...@ime.usp.br wrote: I have been wondering whether it is possible to create a backup system using mtree and rsync. Essentially, the user would create a mtree specification of the source directory and copy it over to the destination directory with rsync. Any changes in the destination could then be detected before restoring with the mtree specification, which should contain strong hashes of the files and should not contain the nlink keyword. A little bit off-topic, but there is a small tool that does something similar to your suggested `mtree` usage, but specifically tailored for backups, `rdup`: http://miek.nl/projects/rdup Although I've not used it myself (I use `rdiff-backup` and on Linux), the idea is pretty similar with what you want to achieve: * you run `rdup` with an old descriptor file plus a target path, and in turn it generates: * a new descriptor file; * a list of files that should be backed up; * you then decide what you do with the list of files to be backed-up (i.e. put them in a `tar`, `rysnc` them to a server, etc.); Hope it helps, Ciprian. schu...@ime.usp.br ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup with mtree and rsync?
No (not directly, except overwriting directories with content), but cpdup can; see man cpdup for details and inspiration. True, but cpdup is not part of the base system. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Backup with mtree and rsync?
I have been wondering whether it is possible to create a backup system using mtree and rsync. Essentially, the user would create a mtree specification of the source directory and copy it over to the destination directory with rsync. Any changes in the destination could then be detected before restoring with the mtree specification, which should contain strong hashes of the files and should not contain the nlink keyword. The problem is that mtree would be too slow. It would recompute the hashes of big files even when they did not change from the last backup. Therefore, I would like to ask if there is an easy way to accomplish the following. Let a mtree specification of a directory from a certain point in the past be given. Also, assume that a (regular) file below that directory has not changed if its current modification time (mtime) equals its modification time in the past specification. Produce as output the new mtree specification for the directory without reading these files. This is somewhat like rsync does to perform incremental backups. P.S.: As an aside, is there an utility in the base system that can reproduce the behavior of `rsync --delete -a dir0/ dir1/`? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
new backup server file system options
Hi all, I am building a new freebsd fileserver to use for backups, will be using 2 disk raid mirroring in a HP microserver n40l. I have gone through some of the documentation and would like to know what file systems to choose. According to the docs, ufs is suggested for the system partitions but someone on the freebsd irc channel suggested using zfs for the rootfs as well Are there any disadvantages of using zfs for the whole system rather than going with ufs for the system files and zfs for the user data? -- Kind regards, Yudi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
ZFS info WAS: new backup server file system options
On Dec 21, 2012, at 7:49 AM, yudi v wrote: I am building a new freebsd fileserver to use for backups, will be using 2 disk raid mirroring in a HP microserver n40l. I have gone through some of the documentation and would like to know what file systems to choose. According to the docs, ufs is suggested for the system partitions but someone on the freebsd irc channel suggested using zfs for the rootfs as well Are there any disadvantages of using zfs for the whole system rather than going with ufs for the system files and zfs for the user data? First a disclaimer, I have been working with Solaris since 1995 and managed lots of data under ZFS, I have only been working with FreeBSD for about the past 6 months. UFS is clearly very stable and solid, but to get redundancy you need to use a separate volume manager. ZFS is a completely different way of thinking about managing storage (not just a filesystem). I prefer ZFS for a number of reasons: 1) End to end data integrity through checksums. With the advent of 1 TB plus drives, the uncorrectable error rate (typically 10^-14 or 10^-15) means that over the life of any drive you *are* now likely to run into uncorrectable errors. This means that traditional volume managers (which rely on the drive reporting an bad reads and writes) cannot detect these errors and bad data will be returned to the application. 2) Simplicity of management. Since the volume management and filesystem layers have been combined, you don't have to manage each separately. 3) Flexibility of storage. Once you build a zpool, the filesystems that reside on it share the storage of the entire zpool. This means you don't have to decide how much space to commit to a given filesystem at creation. It also means that all the filesystems residing in that one zpool share the performance of all the drives in that zpool. 4) Specific to booting off of a ZFS, if you move drives around (as I tend to do in at least one of my lab systems) the bootloader can still find the root filesystem under ZFS as it refers to it by zfs device name, not physical drive device name. Yes, you can tell the bootloader where to find root if you move it, but zfs does that automatically. 5) Zero performance penalty snapshots. The only cost to snapshots is the space necessary to hold the data. I have managed systems with over 100,000 snapshots. I am running two production, one lab, and a bunch of VBox VMs all with ZFS. The only issue I have seen is one I have also seen under Solaris with ZFS. Certain kinds of hardware layer faults will cause the zfs management tools (the zpool and zfs commands) to hang waiting on a blocking I/O that will never return. The data continuos to be available, you just can't manage the zfs infrastructure until the device issues are cleared. For example, if you remove a USB drive that hosts a mounted ZFS, then any attempt to manage that ZFS device will hang (zpool export -f zpool name hangs until a reboot). Previously I had been running (at home) a fileserver under OpenSolaris using ZFS and it saved my data when I had multiple drive failures. At a certain client we had a 45 TB configuration built on top of 120 750GB drives. We had multiple redundancy and could survive a complete failure of 2 of the 5 disk enclosures (yes, we tested this in pre-production). There are a number of good writeups on how setup a FreeBSD system to boot off of ZFS, I like this one the best http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE , but I do the zpool/zfs configuration slightly differently (based on some hard learned lessons on Solaris). I am writing up my configuration (and why I do it this way), but it is not ready yet. Make sure you look at all the information here: http://wiki.freebsd.org/ZFS , keeping in mind that lots of it was written before FreeBSD 9. I would NOT use ZFS, especially for booting, prior to release 9 of FreeBSD. Some of the reason for this is the bugs that were fixed in zpool version 28 (included in release 9). -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS info WAS: new backup server file system options
On 2012-12-21 11:28, Arthur Chance wrote: On 12/21/12 14:06, Paul Kraus wrote: On Dec 21, 2012, at 7:49 AM, yudi v wrote: I am building a new freebsd fileserver to use for backups, will be using 2 disk raid mirroring in a HP microserver n40l. I have gone through some of the documentation and would like to know what file systems to choose. According to the docs, ufs is suggested for the system partitions but someone on the freebsd irc channel suggested using zfs for the rootfs as well Are there any disadvantages of using zfs for the whole system rather than going with ufs for the system files and zfs for the user data? First a disclaimer, I have been working with Solaris since 1995 and managed lots of data under ZFS, I have only been working with FreeBSD for about the past 6 months. UFS is clearly very stable and solid, but to get redundancy you need to use a separate volume manager. Slight correction here - you don't need a volume manager (as I understand the term), you'd use the GEOM subsystem, specifically gmirror in this case. See man gmirror for details ZFS is a completely different way of thinking about managing storage (not just a filesystem). I prefer ZFS for a number of reasons: 1) End to end data integrity through checksums. With the advent of 1 TB plus drives, the uncorrectable error rate (typically 10^-14 or 10^-15) means that over the life of any drive you *are* now likely to run into uncorrectable errors. This means that traditional volume managers (which rely on the drive reporting an bad reads and writes) cannot detect these errors and bad data will be returned to the application. 2) Simplicity of management. Since the volume management and filesystem layers have been combined, you don't have to manage each separately. 3) Flexibility of storage. Once you build a zpool, the filesystems that reside on it share the storage of the entire zpool. This means you don't have to decide how much space to commit to a given filesystem at creation. It also means that all the filesystems residing in that one zpool share the performance of all the drives in that zpool. 4) Specific to booting off of a ZFS, if you move drives around (as I tend to do in at least one of my lab systems) the bootloader can still find the root filesystem under ZFS as it refers to it by zfs device name, not physical drive device name. Yes, you can tell the bootloader where to find root if you move it, but zfs does that automatically. 5) Zero performance penalty snapshots. The only cost to snapshots is the space necessary to hold the data. I have managed systems with over 100,000 snapshots. I am running two production, one lab, and a bunch of VBox VMs all with ZFS. The only issue I have seen is one I have also seen under Solaris with ZFS. Certain kinds of hardware layer faults will cause the zfs management tools (the zpool and zfs commands) to hang waiting on a blocking I/O that will never return. The data continuos to be available, you just can't manage the zfs infrastructure until the device issues are cleared. For example, if you remove a USB drive that hosts a mounted ZFS, then any attempt to manage that ZFS device will hang (zpool export -f zpool name hangs until a reboot). Previously I had been running (at home) a fileserver under OpenSolaris using ZFS and it saved my data when I had multiple drive failures. At a certain client we had a 45 TB configuration built on top of 120 750GB drives. We had multiple redundancy and could survive a complete failure of 2 of the 5 disk enclosures (yes, we tested this in pre-production). There are a number of good writeups on how setup a FreeBSD system to boot off of ZFS, I like this one the best http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE , but I do the zpool/zfs configuration slightly differently (based on some hard learned lessons on Solaris). I am writing up my configuration (and why I do it this way), but it is not ready yet. Make sure you look at all the information here: http://wiki.freebsd.org/ZFS , keeping in mind that lots of it was written before FreeBSD 9. I would NOT use ZFS, especially for booting, prior to release 9 of FreeBSD. Some of the reason for this is the bugs that were fixed in zpool version 28 (included in release 9). I would agree with all that. My current system uses UFS filesystems for the base install, and ZFS with a raidz zpool for everything else, but that's only because I started using ZFS in REL 8.0 when it was just out of experimental status, and I didn't want to risk having an unbootable system. (That last paragraph suggests I was wise in that decision.) My next machine I'm specing out now will be pure ZFS so I get the boot environment stuff. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to
portmaster backup package
Hi, what to do with that backup package (-b option) after installation of new port failed ? jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: portmaster backup package
On Mon, 8 Oct 2012 16:18:20 + (UTC), jb wrote: Hi, what to do with that backup package (-b option) after installation of new port failed ? Install it with pkg_add? :-) -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
firefox menu backup color/transparency
After a upgrading to 15, and subsequently firefox-15.0.1,1 this morning, all menus (menu bar, right click context menu, etc) have a fully transparent background rendering it nearly useless. I've tried starting with a new profile and that doesn't fix it as well as with qt4/gtk2 to now avail. The system is running kde-4.8.4 desktop. Anyone have any suggestions on how to fix? I was planning to migrate to chrome anyway but would like to have a functional firefox. The last 14.x release in the ports tree worked fine. -Kelsey ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Mounting raw disk backup file.
HI, I had a drive fail recently, it was working fine until I rebooted. After that the partition map was corrupt and I can't mount either partition on the disk. So I made a copy of the whole disk using dd to an old USB drive. There were several IO errors while dd was copying the disk, so I think the disk is starting to go. I can probably fix the partition table using testdisk, but now that I've got this image file I'd rather work with that instead of the physical disk. I've read the Handbook section on using mdconfig, but that assumes the image file is of a filesystem, not a whole disk. I think I've found instructions for how to do it on linux, but if there's a way to mount it on FreeBSD I'd rather do that. So, any suggestions? Here's what file says about the file: mnavarre@pcbsd-1810] /# file /mnt/ada1_backup /mnt/ada1_backup: x86 boot sector; partition 1: ID=0xa5, active, starthead 1, startsector 63, 167766732 sectors; partition 2: ID=0xa5, starthead 254, startsector 167766795, 144809910 sectors, code offset 0x3c, BSD disklabel And just for grins, what fdisk says about the actual disk: mnavarre@pcbsd-1810] /# fdisk ada1 *** Working on device /dev/ada1 *** parameters extracted from in-core disklabel are: cylinders=310098 heads=16 sectors/track=63 (1008 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=310098 heads=16 sectors/track=63 (1008 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 63, size 167766732 (81917 Meg), flag 80 (active) beg: cyl 0/ head 1/ sector 1; end: cyl 1023/ head 254/ sector 63 The data for partition 2 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 167766795, size 144809910 (70707 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 3 is: UNUSED The data for partition 4 is: UNUSED Thanks, Matt Navarre ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Mounting raw disk backup file.
On Sun, Aug 5, 2012 at 11:12 PM, Matthew Navarre navarre.matt...@gmail.comwrote: I had a drive fail recently, it was working fine until I rebooted. After that the partition map was corrupt and I can't mount either partition on the disk. So I made a copy of the whole disk using dd to an old USB drive. There were several IO errors while dd was copying the disk, so I think the disk is starting to go. I can probably fix the partition table using testdisk, but now that I've got this image file I'd rather work with that instead of the physical disk. I've read the Handbook section on using mdconfig, but that assumes the image file is of a filesystem, not a whole disk. I think I've found instructions for how to do it on linux, but if there's a way to mount it on FreeBSD I'd rather do that. So, any suggestions? Here's what file says about the file: mnavarre@pcbsd-1810] /# file /mnt/ada1_backup /mnt/ada1_backup: x86 boot sector; partition 1: ID=0xa5, active, starthead 1, startsector 63, 167766732 sectors; partition 2: ID=0xa5, starthead 254, startsector 167766795, 144809910 sectors, code offset 0x3c, BSD disklabel Why did you put it in /mnt? That's customarily used for mounting fileystems. Move it ;-) mdconfig -a -t vnode -f /new-path/ada1_backup note the device that's created (probably md0) you can then operate on /dev/md0 as if it were a disk. In particular, you might want to fix the partition map, the label info, etc. You can then fsck the filesystem (presumably something like /dev/md0s1a or /dev/md0a etc). You'll probably need to tell fsck that it's ufs (i.e. fsck -t ufs /dev/md0a ) you can then mount the fs (mount -t ufs /dev/md0a /mnt ) - M ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Mounting raw disk backup file.
On Sun, 5 Aug 2012 23:12:48 -0700, Matthew Navarre wrote: I can probably fix the partition table using testdisk, but now that I've got this image file I'd rather work with that instead of the physical disk. I've read the Handbook section on using mdconfig, but that assumes the image file is of a filesystem, not a whole disk. I think I've found instructions for how to do it on linux, but if there's a way to mount it on FreeBSD I'd rather do that. It depends on _what_ your disk image (typically created by a dd-like utility to make a 1:1 copy of a whole disk) contains. If there are several slices and partitions, each of them can be accessed like it was a physical disk. Let's assume you have /home/you/ada1.dd which is the copy of your former /dev/ada1 disk. You do: # mdconfig -a -t vnode -u 0 -f /home/you/ada1.dd This results in a file /dev/md0 as well as any partitional qualifier specials that might correspond to the disk the copy has been taken from. You can check that with # fdisk /dev/md0 and it should print the same partition table as for the real disk. Now you can access and mount from that disk image, e. g. # mount -t ufs -o ro /dev/md0s1a /mnt as this maybe is the root file system of the 1st slice. Note the use of -o ro in this case. If you have had partitioned your system, you can add those partitions into a fully accessible /mnt tree for that system disk, e. g. # mount -t ufs -o ro /dev/md0s1d /mnt/tmp # mount -t ufs -o ro /dev/md0s1e /mnt/var # mount -t ufs -o ro /dev/md0s1f /mnt/usr # mount -t ufs -o ro /dev/md0s1g /mnt/home Note that unmounting must happen in the reversed order. If there was another file system, e. g. for sharing with Windows stuff, it's also possible to mount it: # mount -t msdosfs -o ro /dev/md0s2 /mnt/win Of course you can access all slices and partitions independently. That should be the best approach for recovering data. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Mounting raw disk backup file.
On Sun, Aug 5, 2012 at 11:28 PM, Michael Sierchio ku...@tenebras.comwrote: On Sun, Aug 5, 2012 at 11:12 PM, Matthew Navarre navarre.matt...@gmail.com wrote: Here's what file says about the file: mnavarre@pcbsd-1810] /# file /mnt/ada1_backup /mnt/ada1_backup: x86 boot sector; partition 1: ID=0xa5, active, starthead 1, startsector 63, 167766732 sectors; partition 2: ID=0xa5, starthead 254, startsector 167766795, 144809910 sectors, code offset 0x3c, BSD disklabel Why did you put it in /mnt? That's customarily used for mounting fileystems. Move it ;-) Heh, the BSD drive with the backup file in on /mnt, the mdconfig node is md1. mdconfig -a -t vnode -f /new-path/ada1_backup note the device that's created (probably md0) you can then operate on /dev/md0 as if it were a disk. In particular, you might want to fix the partition map, the label info, etc. You can then fsck the filesystem (presumably something like /dev/md0s1a or /dev/md0a etc). You'll probably need to tell fsck that it's ufs (i.e. fsck -t ufs /dev/md0a ) you can then mount the fs (mount -t ufs /dev/md0a /mnt ) Thanks, didn't realize that I could use that device node to operate on it like a block device. Of course, the fact that mdconfig makes the system think a file is a block device should have been a clue ;) Now I just need to get the partitions and disklabel figured out. - M OK, MCN ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Mounting raw disk backup file.
On Mon, Aug 6, 2012 at 12:08 AM, Polytropon free...@edvax.de wrote: On Sun, 5 Aug 2012 23:12:48 -0700, Matthew Navarre wrote: I can probably fix the partition table using testdisk, but now that I've got this image file I'd rather work with that instead of the physical disk. I've read the Handbook section on using mdconfig, but that assumes the image file is of a filesystem, not a whole disk. I think I've found instructions for how to do it on linux, but if there's a way to mount it on FreeBSD I'd rather do that. It depends on _what_ your disk image (typically created by a dd-like utility to make a 1:1 copy of a whole disk) contains. If there are several slices and partitions, each of them can be accessed like it was a physical disk. Let's assume you have /home/you/ada1.dd which is the copy of your former /dev/ada1 disk. You do: # mdconfig -a -t vnode -u 0 -f /home/you/ada1.dd This results in a file /dev/md0 as well as any partitional qualifier specials that might correspond to the disk the copy has been taken from. You can check that with Yep. Unfortunately the partition table and disklable are screwed, so md1s(1,2) don't appear in dev. Same with ada1, which is the physical disk. The drive has two primary partitions, both of which had a single UFS file system on them. So now I just need to figure out how to fix the partition table and the disklabel and I should be golden getting the data out. That drive is getting replaced, though. # fdisk /dev/md0 and it should print the same partition table as for the real disk. here's the fdisk output for /dev/ada1, which is the real drive: mnavarre@pcbsd-1810] /# fdisk ada1 *** Working on device /dev/ada1 *** parameters extracted from in-core disklabel are: cylinders=310098 heads=16 sectors/track=63 (1008 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=310098 heads=16 sectors/track=63 (1008 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 63, size 167766732 (81917 Meg), flag 80 (active) beg: cyl 0/ head 1/ sector 1; end: cyl 1023/ head 254/ sector 63 The data for partition 2 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 167766795, size 144809910 (70707 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 3 is: UNUSED The data for partition 4 is: UNUSED Now, the first partition looks sane (size is right, geometry looks right). The second partition, however, looks a bit wrong, since it ends before it starts, but the size is right. And now I've gone and screwed up the disk image file. So, it's math and manpages from here. OK, MCN ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Mounting raw disk backup file.
I had a drive fail recently, it was working fine until I rebooted. After that the partition map was corrupt and I can't mount either partition on the disk. So I made a copy of the whole disk using dd to an old USB drive. There were several IO errors while dd was copying the disk, so I think the disk is starting to go. did you use conv=sync,noerror. if not your backup is quite corrupted if errors were uncorrectable. So, any suggestions? repair partition table, use mdconfig and then you will get /dev/md0 /dev/md0s1... as with real drive ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Mounting raw disk backup file.
On Mon, Aug 6, 2012 at 2:55 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: I had a drive fail recently, it was working fine until I rebooted. After that the partition map was corrupt and I can't mount either partition on the disk. So I made a copy of the whole disk using dd to an old USB drive. There were several IO errors while dd was copying the disk, so I think the disk is starting to go. did you use conv=sync,noerror. if not your backup is quite corrupted if errors were uncorrectable. Yep, did that, so I'll lose some data, but that partition is mostly media and some old MacOS (7/8/9) software, so it'd suck to lose it but wouldn't be catastrophic. The first partition on the disk is more important. The ironic thing is that the drives I was going to transfer this data onto and the external hard drive that was going to be hold the backups are on their way. Murphy strikes when you least expect it. So, any suggestions? repair partition table, use mdconfig and then you will get /dev/md0 /dev/md0s1... as with real drive Yep, it's currently the whole repair partition table bit that's giving me problems. Unfortunately, PC disk geometry is not my forte. I'm hoping I can find my log book from the system this drive was transfered from, which should have both the partition map and the alternate superblocks written down. My wife probably put it somewhere safe ;) OK, MCN ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Invalid GPT backup header
On 2012-07-09 15:11, dweimer wrote: I have had a few virtual machine installations come up with gptboot: invalid GPT backup header error message while booting. (some immediately on first boot after install) They still boot fine, and run without problems, but I would like to find a way to fix the problem. Does anyone know how you can write a new GPT backup header to the partition table. I have even gone to the extent of adding a second virtual disk, using gpart to create a partition table and then add new partitions and setup bootstrap then use cpio to copy data over and switched the drive SCSI IDs within the virtual machines configuration to boot from the second drive instead. Still same error message when booting off the new drive. I did however leave out the size option when creating the last UFS partition so it may have filled to the end and overwrote the backup GPT table. I have searched online for a solution to recreating the backup GPT header from the main header, but I have been unable to find anything. Not sure if I am missing something simple, if no one else has had this problem, or if others are just ignoring it since their system works even with it? All the machines with the problem are VMware hosted machines, some on ESX, and some on VMware Workstations. Just a quick update in case someone else runs into this and finds this thread, this problem maybe related to a quirk of some sort in the older version of VMware workstation I am running on my work laptop its running version 6.5 (can't get boss to approve upgrade as the other admins have just decided its better to test on the ESX servers instead of their local machines, which I don't agree with) I had an urgent need to use a this machine to put a temporary work around in place on another problem and used vmware converter to migrate it from my workstation to the ESX servers. No more invalid backup GPT message when booting the one copy on the ESX servers. I am going to manually copy the files for this VM to the version 8 workstation installation on my home machine and see if that one does or doesn't show the warning on boot, to attempt to better narrow down this cause. -- Thanks, Dean E. Weimer http://www.dweimer.net/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Invalid GPT backup header
I have had a few virtual machine installations come up with gptboot: invalid GPT backup header error message while booting. (some immediately on first boot after install) They still boot fine, and run without problems, but I would like to find a way to fix the problem. Does anyone know how you can write a new GPT backup header to the partition table. I have even gone to the extent of adding a second virtual disk, using gpart to create a partition table and then add new partitions and setup bootstrap then use cpio to copy data over and switched the drive SCSI IDs within the virtual machines configuration to boot from the second drive instead. Still same error message when booting off the new drive. I did however leave out the size option when creating the last UFS partition so it may have filled to the end and overwrote the backup GPT table. I have searched online for a solution to recreating the backup GPT header from the main header, but I have been unable to find anything. Not sure if I am missing something simple, if no one else has had this problem, or if others are just ignoring it since their system works even with it? All the machines with the problem are VMware hosted machines, some on ESX, and some on VMware Workstations. -- Thanks, Dean E. Weimer http://www.dweimer.net/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
My criteria for procedures are: 1. They should minimize the need for additional software beyond the base system as much as reasonably possible. This means not only that I do not good idea. 3. They should provide for incremental backups. do backed up laptops use FreeBSD or have another filesystem. 4. They should provide for the ability to quickly and easily test backup integrity without restoring the backups anywhere, which most likely means some kind of checksum comparisons akin to what rsync provides. 5. They should allow for transferring data from the system to be backed up to the backup server via SSH. there is precisely one tool you need. /usr/ports/net/rsync there is many distros of rsync for windoze if laptops run it. Not sure what actually works but i can check if you wish. i use rsync for backup server, just config is different: my server is behind NAT, and it connects to backed up server with rsync man rsync and read carefully, don't forget -b option it's very useful ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Maybe take a look at lftp, at the mirror option. For basic demands its a compact solution. Cheers herb langhans -- sprachtraining langhans herbert langhans, warschau herbert.raimund[at]gmx.net herbert[at]langhans.com.pl http://www.langhans.com.pl +0048 603 341 441 | jabber:herbs | icq:414500866 | yahoo_im:herbert.raimund ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Maybe take a look at lftp, at the mirror option. For basic demands its a compact solution. try doing backup of things with 1 dirs and million files and certainly you will understand you need rsync. ftp protocol is plain bad for that. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Fri, Jun 22, 2012 at 06:37:17PM -0600, Chad Perrin wrote: On Fri, Jun 22, 2012 at 08:47:40PM +0200, Roland Smith wrote: On Fri, Jun 22, 2012 at 10:09:03AM -0600, Chad Perrin wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. What are the laptops running? FreeBSD, Debian, and/or Ubuntu. There's at least one of each. I apologize for not mentioning that sooner. I had a feeling I'd overlook something. Hmm, I'm not sure that there is _anything_ that meets _all_ your criteria! For backing up complete systems (including boot blocks) I've used Clonezilla Live to good effect. On the several standalone systems I tried it on, it managed around 1 GiB/minute, backing up to a USB HDD. It can also back-up to a ssh, samba or nfs server. Of course this doesn't do incremental backups, and it is GPL. If you don't care about the OS, and just want to back up the user's data, I guess rsync would be the way to go. This in turn will not save the boot block, although you could use dd for that I guess. Roland -- R.F.Smith http://rsmith.home.xs4all.nl/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgph34aKSfOky.pgp Description: PGP signature
Re: backup tools
Hmm, I'm not sure that there is _anything_ that meets _all_ your criteria! rsync meets. It can be a little harder with windoze, with any unix-like OS it will work. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Sat, 23 Jun 2012 09:49:39 +0200 (CEST), Wojciech Puchar wrote: Maybe take a look at lftp, at the mirror option. For basic demands its a compact solution. try doing backup of things with 1 dirs and million files and certainly you will understand you need rsync. In addition to rsync, which is regarded the default tool for the described action, maybe cpdup is worth looking at. It also has the ability to maintain incremental backups (add changes). ftp protocol is plain bad for that. And insecure unless tunneled through some encryption (which might be important when backups appear inside a network with non-trusted participants, or across the Internet). -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
lftp does work incremental. Take a look at Chad's posting again and read what he needs. And of course, ftp via ssh is nothing new ... Cheers herb langhans On Sat, Jun 23, 2012 at 10:22:04AM +0200, Polytropon wrote: On Sat, 23 Jun 2012 09:49:39 +0200 (CEST), Wojciech Puchar wrote: Maybe take a look at lftp, at the mirror option. For basic demands its a compact solution. try doing backup of things with 1 dirs and million files and certainly you will understand you need rsync. In addition to rsync, which is regarded the default tool for the described action, maybe cpdup is worth looking at. It also has the ability to maintain incremental backups (add changes). ftp protocol is plain bad for that. And insecure unless tunneled through some encryption (which might be important when backups appear inside a network with non-trusted participants, or across the Internet). -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... -- sprachtraining langhans herbert langhans, warschau herbert.raimund[at]gmx.net herbert[at]langhans.com.pl http://www.langhans.com.pl +0048 603 341 441 | jabber:herbs | icq:414500866 | yahoo_im:herbert.raimund ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
lftp does work incremental. Take a look at Chad's posting again and read what he needs. And of course, ftp via ssh is nothing new ... still - any ftp client will no go faster than ftp protocol allows. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Sat, Jun 23, 2012 at 11:10:06AM +0200, Wojciech Puchar wrote: lftp does work incremental. Take a look at Chad's posting again and read what he needs. And of course, ftp via ssh is nothing new ... still - any ftp client will no go faster than ftp protocol allows. That's sure. But I think it's an option for the laptops what Chad mentioned. Such scripts for backup are set up in minutes and it happily copies the files to the server. If there are already user accounts on the server, it could be really easy. I think it depends on the scale of the network. Cheers herb langhans ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
At 02:37 23/06/2012, Chad Perrin wrote: On Fri, Jun 22, 2012 at 08:47:40PM +0200, Roland Smith wrote: On Fri, Jun 22, 2012 at 10:09:03AM -0600, Chad Perrin wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. What are the laptops running? FreeBSD, Debian, and/or Ubuntu. There's at least one of each. I apologize for not mentioning that sooner. I had a feeling I'd overlook something. If it must work with all OS and you have no restrictions on network you can: a) activate PXE/WOL on bios b) start the laptop via PXE using a freebsd/linux/whatever_os_you_want_to_use c) use dd piped to rsync to make the backups This way you don't need to install anything on your freebsd ubuntu debian. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
still - any ftp client will no go faster than ftp protocol allows. That's sure. But I think it's an option for the laptops what Chad only if $HOME directly or part of it is copied and nothing more ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
a) activate PXE/WOL on bios b) start the laptop via PXE using a freebsd/linux/whatever_os_you_want_to_use c) use dd piped to rsync to make the backups not really efficient but working. ntfsprogs from ports can be helpful. you may use ntfsmount and access NTFS files directly. if backup is done over fast LAN, ntfsclone -s is useful ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Sat, 23 Jun 2012, Eduardo Morras wrote: At 02:37 23/06/2012, Chad Perrin wrote: On Fri, Jun 22, 2012 at 08:47:40PM +0200, Roland Smith wrote: On Fri, Jun 22, 2012 at 10:09:03AM -0600, Chad Perrin wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. What are the laptops running? FreeBSD, Debian, and/or Ubuntu. There's at least one of each. I apologize for not mentioning that sooner. I had a feeling I'd overlook something. If it must work with all OS and you have no restrictions on network you can: a) activate PXE/WOL on bios b) start the laptop via PXE using a freebsd/linux/whatever_os_you_want_to_use c) use dd piped to rsync to make the backups This way you don't need to install anything on your freebsd ubuntu debian. PXE booting gives a lot of possibilities. I use it to boot Clonezilla to back up Windows systems. That is better than dd, since only used disk blocks are copied. But neither does incremental backups. For FreeBSD and other open operating systems, sysutils/rsnapshot is a possibility. Normally run from cron, could be run manually, or automatically when the backup server is detected. It does incremental copies, is space-efficient (rsync with hard links), and only depends on rsync and Perl. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Wojciech Puchar:
Hmm, I'm not sure that there is _anything_ that meets _all_ your criteria!
rsync meets. It can be a little harder with windoze, with any unix-like OS
it will work.
rsync, or some front-end to rsync, is indeed probably the best option, though
it lacks several of the features that the OP indicates would be desirable.
For several years I've used dirvish to good effect. It's built on rsync and
handles unattended backups over heterogeneous networks quite well. It shares
some of rsync's deficiencies, but for me, its merits (well-structured
simplifications of rsync's ability to exclude files or directories, elegant
handling of backups' expirations) are sufficient to make it a worthy
alternative to naked rsync. The frontend is written in Perl and easily
extended.
By heterogeneous networks I'm afraid I mean ones composed of machines running
unix-like OSs; I've no idea if there's an rsync port to Windows.
Jorge
--
Jorge Luis González list+free...@jorge.cc
http://www.jorge.cc/ * ftp://ftp.jorge.cc/{pub,incoming}
IRC: #vim jl-satyr * XMPP: jl-sa...@jabber.org
GPG KEY - 0x4AD9C195 * ICBM: 42.592627, -72.588859
This email optimized for teletypes.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
what exactly deficiences and requirements not met by rsync are you talking about? simplifications of rsync's ability to exclude files or directories, elegant handling of backups' expirations) are sufficient to make it a worthy alternative to naked rsync. The frontend is written in Perl and easily extended. By heterogeneous networks I'm afraid I mean ones composed of machines running unix-like OSs; I've no idea if there's an rsync port to Windows. there are many. I know people using it ... after they know how useful it is based on my examples. No idea how stable and usable they are. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
PXE booting gives a lot of possibilities. I use it to boot Clonezilla to back up Windows systems. That is better than dd, since only used disk blocks ntfsclone is what you need. for sure simpler. For FreeBSD and other open operating systems, sysutils/rsnapshot is a what is exactly rsnapshot added value to rsync, and what is exactly this fuss about hardlinks? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Wojciech Puchar wrote:
what exactly deficiences and requirements not met by rsync are you talking
about?
Perhaps deficiencies was too strong a word. I think the OP required--or
perhaps desired--a WOL function. I'm not aware of any such capability in rsync
proper. I meant, too, that dirvish, which was the alternative that I
recommended, presents an elegant and easily-comprehended way to manage rsync's
considerable abilities, not that it provides features that can't be managed
directly by rsync.
By heterogeneous networks I'm afraid I mean ones composed of machines
running
unix-like OSs; I've no idea if there's an rsync port to Windows.
there are many. I know people using it ... after they know how useful it
is based on my examples. No idea how stable and usable they are.
Thanks for pointing out that there are Windows ports of rsync, and that you
provide examples of their use. I'm not sure I would entrust my system backups
to them if they come with the disclaimer that you've no idea how stable and
usable they are.
Jorge
--
Jorge Luis González list+free...@jorge.cc
http://www.jorge.cc/ * ftp://ftp.jorge.cc/{pub,incoming}
IRC: #vim jl-satyr * XMPP: jl-sa...@jabber.org
GPG KEY - 0x4AD9C195 * ICBM: 42.592627, -72.588859
This email optimized for teletypes.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
you mean wake on lan? there is wol tool in ports. proper. I meant, too, that dirvish, which was the alternative that I recommended, presents an elegant and easily-comprehended way to manage rsync's considerable abilities, not that it provides features that can't be managed directly by rsync. fine but i really want to manage features directly by specifying a commands. thanks for answer, but i really don't recommend anyone using all in one tools as it's always to have problems with one than with all. there are many. I know people using it ... after they know how useful it is based on my examples. No idea how stable and usable they are. Thanks for pointing out that there are Windows ports of rsync, and that you provide examples of their use. I'm not sure I would entrust my system backups to them if they come with the disclaimer that you've no idea how stable and usable they are. google rsync for windows. It is not a danger if you run this no really sure tools from windoze and you see whether it finished work properly or not. syncback works fine and is used by me. but it is not high performance, it can use only FTP or windows share destination. for backing up my documents it is fine anyway. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Thanks for pointing out that there are Windows ports of rsync, and that you provide examples of their use. I'm not sure I would entrust my system backups to them if they come with the disclaimer that you've no idea how stable and usable they are. http://justinsomnia.org/2007/02/how-to-regularly-backup-windows-xp-to-ubuntu-using-rsync/ might be useful for you, after you ignore all this linux style sudo nonsense. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Sat, Jun 23, 2012 at 09:46:02AM -0400, Jorge Luis Gonzalez wrote: Wojciech Puchar wrote: what exactly deficiences and requirements not met by rsync are you talking about? Perhaps deficiencies was too strong a word. I think the OP required--or perhaps desired--a WOL function. I'm not aware of any such capability in rsync proper. I meant, too, that dirvish, which was the alternative that I recommended, presents an elegant and easily-comprehended way to manage rsync's considerable abilities, not that it provides features that can't be managed directly by rsync. Actually, a Wake-On-LAN feature is not at all necessary for me in this case. It's a simple enough task to just trigger a backup manually at the command line via a script that automates the process. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Sat, Jun 23, 2012 at 11:17:36AM +0200, herbert langhans wrote: On Sat, Jun 23, 2012 at 11:10:06AM +0200, Wojciech Puchar wrote: lftp does work incremental. Take a look at Chad's posting again and read what he needs. And of course, ftp via ssh is nothing new ... still - any ftp client will no go faster than ftp protocol allows. That's sure. But I think it's an option for the laptops what Chad mentioned. Such scripts for backup are set up in minutes and it happily copies the files to the server. If there are already user accounts on the server, it could be really easy. I think it depends on the scale of the network. It does appear to meet my needs, at first glance, with any capabilities it does not already have that I might need easily scripted. I'm having a difficult time finding any reference to licensing, though. Matt Dillon's explanation of cpdup suggests it is probably some kind of BSD-licensed, given its inclusion in DragonFly BSD base utilities, but that's not *necessarily* the case. Reference: http://apollo.backplane.com/FreeSrc/ I'm going to try emailing Dillon for clarification, too. In any case, I'll take a closer look at cpdup. Thanks for bringing it to my attention. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Actually, a Wake-On-LAN feature is not at all necessary for me in this case. It's a simple enough task to just trigger a backup manually at the command line via a script that automates the process. still. a separate wol tool is available in ports. You may easily construct shell script that will execute it, wait a bit, check out if server booted with ping, then wait a bit more (so inetd or rsyncd started) then run rsync. Unix philosophy means have one program to do think well, not to do everything. This is what make me an exclusive unix fanatics. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
backup tools
I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. I'm trying to decide on what tools to use for managing backups. In the past I have used rsync, which has worked reasonably well, but fails one of my desired criteria for the new backup procedures, and is less than ideal for others. My criteria for procedures are: 1. They should minimize the need for additional software beyond the base system as much as reasonably possible. This means not only that I do not want to have to specify the installation of a bunch of stuff, but also that I do not want a bunch of dependencies pulled in with something I choose to install (if anything). Ideally, I should be able to do this with just the base system, though that seems unlikely at this point. 2. They should require only copyfree licensed or public domain tools -- no copyleft licensed tools, no proprietary licensed tools, no noncommercial or nonderivative licensed tools, and no permissively licensed tools where the license comes with annoying restrictions such as the Apache License requirements for specific bookkeeping procedures. I might bend on the requirement for non-copyfree permissive licenses if I have to, but I'd rather not; bending on any of the others would probably involve just giving up and going back to rsync. 3. They should provide for incremental backups. 4. They should provide for the ability to quickly and easily test backup integrity without restoring the backups anywhere, which most likely means some kind of checksum comparisons akin to what rsync provides. 5. They should allow for transferring data from the system to be backed up to the backup server via SSH. 6. They should use tools as simple as possible, preferably command line tools. 7. There should be documentation somewhere out there for how to set something like this up, someone willing to help me figure out how to get it set up, or an obvious path to setting it up so that I do not spend a week just figuring it all out, if at all possible. 8. They should preferably not require creating a local archive on the laptop before copying to the backup server if it can reasonably be avoided, so that a big chunk of empty HDD space will not need to be maintained for backups to work. Any help figuring out what tools would work for these purposes would be appreciated. I might be able to make exceptions for some parts of this if there are suitable alternative approaches. Thanks in advance for any help I can get in figuring this out. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Fri, Jun 22, 2012 at 10:09:03AM -0600, Chad Perrin wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. What are the laptops running? Roland -- R.F.Smith http://rsmith.home.xs4all.nl/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpzbtvy014nJ.pgp Description: PGP signature
Re: backup tools
On Fri, Jun 22, 2012 at 08:47:40PM +0200, Roland Smith wrote: On Fri, Jun 22, 2012 at 10:09:03AM -0600, Chad Perrin wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. What are the laptops running? FreeBSD, Debian, and/or Ubuntu. There's at least one of each. I apologize for not mentioning that sooner. I had a feeling I'd overlook something. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Fri, Jun 22, 2012 at 11:09 AM, Chad Perrin per...@apotheon.com wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. I'm trying to decide on what tools to use for managing backups. In the past I have used rsync, which has worked reasonably well, but fails one of my desired criteria for the new backup procedures, and is less than ideal for others. One's I use or have used: sysutils/rdiff-backup sysutils/tarsnap misc/amanda-server -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
On Fri, Jun 22, 2012 at 08:14:34PM -0500, Adam Vande More wrote: On Fri, Jun 22, 2012 at 11:09 AM, Chad Perrin per...@apotheon.com wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. I'm trying to decide on what tools to use for managing backups. In the past I have used rsync, which has worked reasonably well, but fails one of my desired criteria for the new backup procedures, and is less than ideal for others. One's I use or have used: sysutils/rdiff-backup sysutils/tarsnap misc/amanda-server Unfortunately, one of those is GPL, another is subject to proprietary licensing, and the last has a bunch of (otherwise unnecessary on the server) GNU project dependencies. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: backup tools
Bacula is the tool Enviado desde mi iPod El 22/06/2012, a las 8:31 p.m., Chad Perrin per...@apotheon.com escribió: On Fri, Jun 22, 2012 at 08:14:34PM -0500, Adam Vande More wrote: On Fri, Jun 22, 2012 at 11:09 AM, Chad Perrin per...@apotheon.com wrote: I'm setting up a new backup server using FreeBSD. It will be used for backing up laptops, which will not be connected to the network by any kind of schedule, so backups will be initiated manually rather than by cron or other scheduled procedures. I'm trying to decide on what tools to use for managing backups. In the past I have used rsync, which has worked reasonably well, but fails one of my desired criteria for the new backup procedures, and is less than ideal for others. One's I use or have used: sysutils/rdiff-backup sysutils/tarsnap misc/amanda-server Unfortunately, one of those is GPL, another is subject to proprietary licensing, and the last has a bunch of (otherwise unnecessary on the server) GNU project dependencies. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org Este mensaje y/o sus anexos son para uso exclusivo de su destinatario intencional y puede contener información legalmente protegida por ser confidencial. Si usted no es el destinatario intencional del mensaje por favor infórmenos de inmediato y elimínelo, así como sus anexos. Igualmente, le comunicamos que cualquier retención, revisión no autorizada, distribución, divulgación, reenvío, copia, impresión, reproducción, o uso indebido de este mensaje y/o sus anexos, está estrictamente prohibida y sancionada legalmente. EDATEL S.A. no se hace responsable en ningún caso por daños derivados de la recepción del presente mensaje. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Sunday 10 June 2012 23:14:57 Ronald F. Guilmette wrote: Well, nevermind about that. I get the general idea, i.e. that dumping at level N causes dumping of everything that has changed since the last dump at level N-1. A point to be aware of is that if you restore from a full backup followed by one or more incrementals then you will restore ALL files which were present when each dump was made - including any files which have been intentionally deleted since the dump was created. This isn't normally a problem but there might be some obscure situations where it could be. -- Mike Clarke ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Monday 11 June 2012 00:03:59 Daniel Feenberg wrote: It does occur to me that /etc is not a felicitous place to keep this information, but given the desirability of dumping filesystems in read only state, placing the dump dates in the filesystem itself isn't feasible. Dumping with the -L option creates a temporary read only snapshot which is used as the source for the backup. This enables you to safely backup a live filesystem, More background on snapshots at http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/snapshots.html -- Mike Clarke ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
Hello. 2012/06/09 19:30:53 -0700 Ronald F. Guilmette r...@tristatelogic.com = To Arthur Chance : RFG Thank you Arthur, and yes, trying to back up a partition that's currently RFG mounted r/w using dd will almost certainly not produce the desired results. You can make snapshot to back up rw-mounted volume with dd. dump(8)ing rw-mounted ufs makes a snapshot behind the scenes. -- Peter Vereshagin pe...@vereshagin.org (http://vereshagin.org) pgp: A0E26627 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Sunday 10 June 2012 03:30:53 Ronald F. Guilmette wrote: I don't care to take own my system to make backups... and don't believe that I should have to do so, and thus, this is one of the reasons why I would prefer to use something like cpio. Also, I don't like backups taking longer than absolutely necessary, and this is why I am specifically _not_ attracted to either the dd solution or to dump/restore, Not an immediate solution but have you considered switching from UFS to ZFS ? If you have sufficient memory and CPU power then this might be worth the effort. Creating ZFS snapshots and backing them up incrementally with zfs send | zfs receive should be very quick. -- Mike Clarke ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
From owner-freebsd-questi...@freebsd.org Sat Jun 9 21:33:57 2012 To: Arthur Chance free...@qeng-ho.org Date: Sat, 09 Jun 2012 19:30:53 -0700 From: Ronald F. Guilmette r...@tristatelogic.com Cc: freebsd-questions@freebsd.org Subject: Re: Making a bootable backup (hard)disk... how? In message 4fd38b9a.4010...@qeng-ho.org, Arthur Chance free...@qeng-ho.org wrote: There's a BFI (brute force and ignorance) way of doing it in the base system - dd. Provided your system disk is quiescent (ideally when running from a live CD or all partitions mounted read-only, otherwise pray to the deity of your choice) and the backup disk is a) at least as large as the system disk, and b) has the same sector size, then a simple dd from the system disk to the backup should work. Thank you Arthur, and yes, trying to back up a partition that's currently mounted r/w using dd will almost certainly not produce the desired results. And of course, as you correctly note also, the target partition had best be at least as big as the source (and perhaps even identical in size). I don't care to take own my system to make backups... and don't believe that I should have to do so, and thus, this is one of the reasons why I would prefer to use something like cpio. Also, I don't like backups taking longer than absolutely necessary, and this is why I am specifically _not_ attracted to either the dd solution or to dump/restore, because as I understand it, with either of these methods you end up copying perhaps a metric buttload worth of unallocated free disk space. (I would prefer not to do that. It just seems wasteful... of time, if nothing else.) Dump does *NOT* copy 'unallocated free disk space'. It does a directory- based traversal. And copies only the contents of the files encountered. does not copy 'deleted but still open' files, nor does it include deleted directory entries in the copy of the directory. dump _can_ be used on a r/w-mounted filesystem. It is smart enough to make multiple passes, note files that have changed since the prior pass, and re-copy those changed files into the archive. As long as changes are relatvely 'infrequent', and to 'small' files, dump will generally manage to outrun the changes. Since dump works through the O/S, it also catches data and/or meta-data changes that are buffered/cached in O/S memory that have not actually made it to disk, yet. Using dump on a mounted, active, writable filesystem is not recommended, but it -does- work adequately, assuming the write activity on the filesystem is low. *IF* the filesystem is fairly full, with source and dest arranged such that they do not saturate I/O controller capabilities, 'dd' (or, preferably 'ddd' which is a double-buffered drop-in replacement) with large buffers (several times the size of the 'on disk' cache'), _is_ the fastest way to make a backup of the data. Also, in one case, one of my partitions has one directory that contains a really massive amount of stuff, and I specifically _don't_ need any of that particular stuff (in that one directory) backed up. So again, I'm looking at tar or cpio or perhaps pax. (Of course cpio is more full-featured than tar, and I don't really know anything about pax, so that leaves me with cpio.) Pax (the portable archiver) is the new, improved, with miracle whitener cpio -- does everthing tradtional cpio does, and more. it will read/write four different (and mutually incompatable) 'cpio' formats, and two different 'tar' formats. One can replace pretty much any use of 'cpio' or 'tar' (or their cousins) with pax. It _is_ worth investigating. when. (Sigh. The good old daze when men were men, and the bits ran scared!) Moral of the story is that sometimes it actually does pay to be a smartass. Regards, rfg P.S. It really is a Damn Shame[tm] that nobody ever hacked FreeBSD cpio to make it be able to copy (a) the extra file flag/mode bits and/or (b) file ACLs and/or (c) file attributes. If pax doesn't already support extended flags, it would be a relatively trivial hack to get it to do so. Ditto for ACLs. pax uses libarchive (man 3 archive) which has the capabilities for handling those things in the description of an archive entry. One would probably want to declare it as a separate archive 'type' -- to support auto-detect of the inclusion of that 'extended' metadata. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
In message alpine.bsf.2.00.1206092039260.71...@wonkity.com, you wrote: On Sat, 9 Jun 2012, Ronald F. Guilmette wrote: Also, I don't like backups taking longer than absolutely necessary, and this is why I am specifically _not_ attracted to either the dd solution or to dump/restore, because as I understand it, with either of these methods you end up copying perhaps a metric buttload worth of unallocated free disk space. No, that is one of the biggest advantages of dump over dd. dump knows UFS, and only copies occupied sectors and needed information. OK. I obviously got the Wrong Impression about dump/restore. Thanks for setting me straight. (I got the Wrong Impression, I think, because I have read assertions like ...dump backs up at the filesystem block level What does that mean exactly? Use of the term block level in this context makes me think of something operating along the lines of dd.) Also, in one case, one of my partitions has one directory that contains a really massive amount of stuff, and I specifically _don't_ need any of that... backed up. Directories and files can be skipped with the nodump flag. I am in your debt again. I really will start looking at dump/restore. P.S. It really is a Damn Shame[tm] that nobody ever hacked FreeBSD cpio to make it be able to copy (a) the extra file flag/mode bits and/or (b) file ACLs and/or (c) file attributes. A quick search shows dump should support ACLs. The other stuff is also there. Try it. http://www.wonkity.com/~wblock/docs/html/backup.html THANK YOU! I am finding the above document very helpful. It really never even occured to me that one could pipe the output of dump directly to restore. (Obviously, that pairing creates something very much along the lines of cpio's -p option... which is one thing that I _am_ already familiar with.) Now that I've read your helpful document, I do have three short follow-up questions... 1) In your example under the heading Copying Filesystems, the second shell command line shown is: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) Is that correct? Shouldn't it actually be this instead? dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt/usr restore -ruf -) I mean if the goal is to create a whole backup under /mnt that looks just like the whole original system, then shouldn't ``restoration'' of the /usr part of the original system go into /mnt/usr? 2) Towards the end of your document you mention rsync. Assuming that I have made a backup of my entire /usr partition (using dumprestore) into /mnt/usr and that at midnight every night from now on I want to simply refresh that and bring it up to date with the current contents of my actual /usr partition, what is the most proper way to do this? Should I use rsync for that? Or should I use dumprestore again? If that latter, then how exactly does that work? I mean if I do the pipeline from dump to restore as you have shown in your examples in your Copying Filesystems section, then what must I do in order prevent dump from dumping files that haven't changed? (And likewise, how do I prevent restore from trying to restore files under /mnt/usr that have not changed? Or is that answer to that question that I simply have to do the first thing, i.e. force dump not to dump any of the unchanged files?) 3) Assuming that I have a second disk to which I plan (now) to use dumprestore to make periodic copies of all of my ``normal'' (non-backup) filesystems onto. Assuming also that I've already installed the FreeBSD boot loader into the boot block on this second disk, and that I've already run fdisk bsdlabel on it so as to set up all of the partitioning to be essentially identical to my ``main'' system disk. Assuming that all of this is the case, by using dumprestore as you have shown in your document under the heading Copying Filesystems, will I have succeded at doing what I was first asking about in this thread, i.e. will I have successfully created a _bootable_ mirror of my main system disk... something that I can just plug in and go with, when and if disaster strikes and my main system disk suffers a horrifying head crash, you know, the day after the manufacturer's warranty period is up? Regards, rfg ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Sun, 10 Jun 2012, Ronald F. Guilmette wrote: (I got the Wrong Impression, I think, because I have read assertions like ...dump backs up at the filesystem block level What does that mean exactly? Use of the term block level in this context makes me think of something operating along the lines of dd.) Rather than reading just the contents of files, dump operates at a lower level, backing up all of the blocks used by a filesystem. 1) In your example under the heading Copying Filesystems, the second shell command line shown is: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) Is that correct? Shouldn't it actually be this instead? dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt/usr restore -ruf -) I mean if the goal is to create a whole backup under /mnt that looks just like the whole original system, then shouldn't ``restoration'' of the /usr part of the original system go into /mnt/usr? The mount command above that is mount /dev/da0s1f /mnt The f indicates it's a split filesystem layout, where /, /var, /tmp, and /usr are separate filesystems in different partitions. That's a traditional layout in FreeBSD. bsdinstall in FreeBSD 9 creates a single filesystem, / . Restoring a dump of a /usr filesystem would go to /mnt/usr in that case. (That would mean you are converting from the split layout to a single filesystem, because with a combined filesystem you can't easily back up just /usr with dump. dump does not cross filesystems when making a backup.) 2) Towards the end of your document you mention rsync. Assuming that I have made a backup of my entire /usr partition (using dumprestore) into /mnt/usr and that at midnight every night from now on I want to simply refresh that and bring it up to date with the current contents of my actual /usr partition, what is the most proper way to do this? Should I use rsync for that? Or should I use dumprestore again? If that latter, then how exactly does that work? I mean if I do the pipeline from dump to restore as you have shown in your examples in your Copying Filesystems section, then what must I do in order prevent dump from dumping files that haven't changed? (And likewise, how do I prevent restore from trying to restore files under /mnt/usr that have not changed? Or is that answer to that question that I simply have to do the first thing, i.e. force dump not to dump any of the unchanged files?) See the dump man page about dump levels. I have not tried a dump level higher than zero with a dump|restore copy, but expect it to work. rsync will do it, and would be my choice for that; start with -aH for options, and --exclude might be used for some directories that don't need to be copied. Also see the sysutils/rsnapshot port for an interesting snapshot use. Combined with a bootable FreeBSD like mfsBSD (http://mfsbsd.vx.sk/), that might be a workable alternative to creating a bootable backup. 3) Assuming that I have a second disk to which I plan (now) to use dumprestore to make periodic copies of all of my ``normal'' (non-backup) filesystems onto. Assuming also that I've already installed the FreeBSD boot loader into the boot block on this second disk, and that I've already run fdisk bsdlabel on it so as to set up all of the partitioning to be essentially identical to my ``main'' system disk. Assuming that all of this is the case, by using dumprestore as you have shown in your document under the heading Copying Filesystems, will I have succeded at doing what I was first asking about in this thread, i.e. will I have successfully created a _bootable_ mirror of my main system disk... something that I can just plug in and go with, when and if disaster strikes and my main system disk suffers a horrifying head crash, you know, the day after the manufacturer's warranty period is up? With some tuning, yes. /etc/fstab mountpoints often won't match when such a drive is connected to a different port or new system. Use gpart(8) GPT labels or tunefs(8) filesystem labels. Likewise with the Ethernet board, so see rc.conf(5) about ifconfig_DEFAULT. gpart and GPT are easier than fdisk and bsdlabel. I really need to flip that disk setup article upside down, with gpart first as the new goodness. See also: http://www.wonkity.com/~wblock/docs/html/labels.html http://www.wonkity.com/~wblock/docs/html/disksetup.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
In message alpine.bsf.2.00.1206100543280.75...@wonkity.com, Warren Block wbl...@wonkity.com wrote: On Sun, 10 Jun 2012, Ronald F. Guilmette wrote: 1) In your example under the heading Copying Filesystems, the second shell command line shown is: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) Is that correct? Shouldn't it actually be this instead? dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt/usr restore -ruf -) ... The mount command above that is mount /dev/da0s1f /mnt Ah! OK. I failed to notice that. Yes, that clarifies things entirely. ... I mean if I do the pipeline from dump to restore as you have shown in your examples in your Copying Filesystems section, then what must I do in order prevent dump from dumping files that haven't changed? (And likewise, how do I prevent restore from trying to restore files under /mnt/usr that have not changed? Or is that answer to that question that I simply have to do the first thing, i.e. force dump not to dump any of the unchanged files?) See the dump man page about dump levels. OK, I'm looking at it, and to be honest, the subject of dump levels is not covered at all well there. (Neither does it seem to be covered well in other online documents, found via Google, that purport to describe dump restore.) The reference to modified Tower of Hanoi algorithm and the subsequent suggested dumplevel sequence are especially opaque and perplexing. Well, nevermind about that. I get the general idea, i.e. that dumping at level N causes dumping of everything that has changed since the last dump at level N-1. What I don't understand (and what I wish someone would enlighten me about) is just this: It would seem that in order to implement these dump levels, dump must be keeping a record somewhere, for each file in the filesystem, of the level at which that file was last dumped. But where is this infor- mation stored, exactly?? I won't be able to sleep until I know. (Aside: One of the reasons I hate Windoze is that everything is hidden. One of the reasons I love UNIX is that everything is out in the open, which is to say documented. But even after reading the dump(1) man page, it ain't in the least bit obvious to me where dump is recording the last dump level for each node in the filesystem. This info must be stored SOMEWHERE. But where?) I have not tried a dump level higher than zero with a dump|restore copy, but expect it to work. rsync will do it, and would be my choice for that; start with -aH for options, and --exclude might be used for some directories that don't need to be copied. Humm... well, I am reminded of all of the reasons why I have been convinced (now) to use dump/restore, specifically the security of knowing that _all_ info will be copied across, in particular (a) extra mode bits/flags and (b) ACLs and (c) file attributes. (This is not to even mention correct handling of other obscure filesystem things like, e.g. sparse files.) So the obvious question is: If I am going to use rsync to keep my backup disk up-to-date, does rsync handle all of these additional small but important filesystem details properly too? If not, then I had best just stick with dump/restore, and use the dumplevels. Also see the sysutils/rsnapshot port for an interesting snapshot use. Combined with a bootable FreeBSD like mfsBSD (http://mfsbsd.vx.sk/), that might be a workable alternative to creating a bootable backup. Thanks. I've installed that port and I am looking at the man page, but I believe that I do still want to reach my goal of just having a (recovery) hard disk that I can just boot directly from. 3) Assuming that I have a second disk to which I plan (now) to use dumprestore to make periodic copies of all of my ``normal'' (non-backup) filesystems onto. Assuming also that I've already installed the FreeBSD boot loader into the boot block on this second disk, and that I've already run fdisk bsdlabel on it so as to set up all of the partitioning to be essentially identical to my ``main'' system disk. Assuming that all of this is the case, by using dumprestore as you have shown in your document under the heading Copying Filesystems, will I have succeded at doing what I was first asking about in this thread, i.e. will I have successfully created a _bootable_ mirror of my main system disk... With some tuning, yes. /etc/fstab mountpoints often won't match when such a drive is connected to a different port or new system. Right. There's that. So I will need to diddle the /etc/fstab in the backup. Use gpart(8) GPT labels... Ummm... I don't know pooh about this, but I am reading about it now, and the idea make me _very_ nervous. Apparently, `MBR' is traditional, and I am very much of a traditionalist. (Mostly I find that I only get into trouble when I try to buck pre-existing conventions.) or tunefs(8) filesystem labels. I guess you must mean the -L volname option. I'm not at all sure
Re: Making a bootable backup (hard)disk... how?
Warren? Just a couple more quick questions. You recommend: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) I'm real curious about you suggestions for the -C and -b values. I have what amounts to a personal workstation. Yea, OK, it is running mail, web, and FTP servers also, but fundementailly, it is not that busy most of the time. And it's got 4GB of main installed. On average, I suspect that I ain't even using half of that. Given all that, why shouldn't I specify (e.g.): -C512 -b1024 ? Wouldn't that all make the dump go faster? Remember, I'm just gonna do like in your examples and pipe the output of dump to restore, so whereas if I was going to tape (or even to a .dump disk file), -b1024 would be rather wasteful, in my case all I am wasting is a bit of kernel time (for a bit more stuff being passed through the pipe). Regards, rfg ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Sun, 10 Jun 2012, Ronald F. Guilmette wrote: In message alpine.bsf.2.00.1206100543280.75...@wonkity.com, Warren Block wbl...@wonkity.com wrote: On Sun, 10 Jun 2012, Ronald F. Guilmette wrote: 1) In your example under the heading Copying Filesystems, the second shell command line shown is: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) Is that correct? Shouldn't it actually be this instead? dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt/usr restore -ruf -) ... The mount command above that is mount /dev/da0s1f /mnt Ah! OK. I failed to notice that. Yes, that clarifies things entirely. ... I mean if I do the pipeline from dump to restore as you have shown in your examples in your Copying Filesystems section, then what must I do in order prevent dump from dumping files that haven't changed? (And likewise, how do I prevent restore from trying to restore files under /mnt/usr that have not changed? Or is that answer to that question that I simply have to do the first thing, i.e. force dump not to dump any of the unchanged files?) See the dump man page about dump levels. OK, I'm looking at it, and to be honest, the subject of dump levels is not covered at all well there. (Neither does it seem to be covered well in other online documents, found via Google, that purport to describe dump restore.) The reference to modified Tower of Hanoi algorithm and the subsequent suggested dumplevel sequence are especially opaque and perplexing. It's diffs. One complete copy to start, then a partial dump, just the files that have changed since the first. And then another partial, just the files that have changed since the second. Start over with a full dump every so often, because that reduces the amount of backups that have to be restored until you get to the final version of the file. As far as I can recall, I've always used level zero. What I don't understand (and what I wish someone would enlighten me about) is just this: It would seem that in order to implement these dump levels, dump must be keeping a record somewhere, for each file in the filesystem, of the level at which that file was last dumped. But where is this infor- mation stored, exactly?? I won't be able to sleep until I know. (Aside: One of the reasons I hate Windoze is that everything is hidden. One of the reasons I love UNIX is that everything is out in the open, which is to say documented. But even after reading the dump(1) man page, it ain't in the least bit obvious to me where dump is recording the last dump level for each node in the filesystem. This info must be stored SOMEWHERE. But where?) Sorry, I've never checked. File dates, maybe. The source for dump would be the place to look. rsync will do it, and would be my choice for that; start with -aH for options, and --exclude might be used for some directories that don't need to be copied. Humm... well, I am reminded of all of the reasons why I have been convinced (now) to use dump/restore, specifically the security of knowing that _all_ info will be copied across, in particular (a) extra mode bits/flags and (b) ACLs and (c) file attributes. (This is not to even mention correct handling of other obscure filesystem things like, e.g. sparse files.) So the obvious question is: If I am going to use rsync to keep my backup disk up-to-date, does rsync handle all of these additional small but important filesystem details properly too? If not, then I had best just stick with dump/restore, and use the dumplevels. It does many of those things, enough that many people use it for backup. Right. There's that. So I will need to diddle the /etc/fstab in the backup. Use gpart(8) GPT labels... Ummm... I don't know pooh about this, but I am reading about it now, and the idea make me _very_ nervous. Apparently, `MBR' is traditional, and I am very much of a traditionalist. (Mostly I find that I only get into trouble when I try to buck pre-existing conventions.) or tunefs(8) filesystem labels. I guess you must mean the -L volname option. I'm not at all sure that I see how that would help anything. When a mountpoint is /dev/gpt/myrootfs, it doesn't matter whether the drive shows up as ada0 or ad4 or da2, the mount still works. Likewise with the Ethernet board, so see rc.conf(5) about ifconfig_DEFAULT. Something else (the Handbook?) mentioned something about the ethernet card(s) in the context also. I just don't get it. Why would I even need to think about that in this context? My current system disk is alive well and already has all of the proper ifconfig stuff in /etc/rc.conf to make the local ethernet card work properly on my network. I am engaged in making a _verbatim_ backup of my main system disk to another hard disk that, in an emergency, would be plugged in to the exact same system (with the exact same ethernet card) and if I have made the verbatim backup properly, then it also will have the proper
Re: Making a bootable backup (hard)disk... how?
On Sun, 10 Jun 2012, Ronald F. Guilmette wrote: What I don't understand (and what I wish someone would enlighten me about) is just this: It would seem that in order to implement these dump levels, dump must be keeping a record somewhere, for each file in the filesystem, of the level at which that file was last dumped. But where is this infor- mation stored, exactly?? I won't be able to sleep until I know. Only the dates of the levels of backup are stored, in /etc/dumpdates. Then the fact that a file has been dumped is inferred by comparing the file's last mod date with the dates in /etc/dumpdates. See the -T and -u options of the dump man page where this is implied but perhaps not actually stated. It does occur to me that /etc is not a felicitous place to keep this information, but given the desirability of dumping filesystems in read only state, placing the dump dates in the filesystem itself isn't feasible. daniel feenberg ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
Ronald F. Guilmette r...@tristatelogic.com wrote: Warren Block wbl...@wonkity.com wrote: On Sun, 10 Jun 2012, Ronald F. Guilmette wrote: ... I mean if I do the pipeline from dump to restore as you have shown in your examples in your Copying Filesystems section, then what must I do in order prevent dump from dumping files that haven't changed? (And likewise, how do I prevent restore from trying to restore files under /mnt/usr that have not changed? Or is that answer to that question that I simply have to do the first thing, i.e. force dump not to dump any of the unchanged files?) See the dump man page about dump levels. OK, I'm looking at it, and to be honest, the subject of dump levels is not covered at all well there. (Neither does it seem to be covered well in other online documents, found via Google, that purport to describe dump restore.) The reference to modified Tower of Hanoi algorithm and the subsequent suggested dumplevel sequence are especially opaque and perplexing. Well, nevermind about that. I get the general idea, i.e. that dumping at level N causes dumping of everything that has changed since the last dump at level N-1. That's about all -really- need to know. The sequencing stuff is just a recommendation on a way to do multiple levels of incremental backups. What I don't understand (and what I wish someone would enlighten me about) is just this: It would seem that in order to implement these dump levels, dump must be keeping a record somewhere, for each file in the filesystem, of the level at which that file was last dumped. But where is this infor- mation stored, exactly?? I won't be able to sleep until I know. First, your assumption is incorrect. *grin* All that dump stores is the timestamp of the completion of the last dump at each level. As for where that information is stored. RTFM applies. the 'files' section. By default '/etc/dumpdates', but modifiable by the -D option. (Aside: One of the reasons I hate Windoze is that everything is hidden. One of the reasons I love UNIX is that everything is out in the open, which is to say documented. But even after reading the dump(1) man page, it ain't in the least bit obvious to me where dump is recording the last dump level for each node in the filesystem. This info must be stored SOMEWHERE. But where?) Wrong assumptions lead to looking for the wrong things. grin Why store the date for every node, when storing the date of the dump sufficies? (assuming you're consisently using the same dump command for dumps recorded to the same 'dumpfile', that is, and using different dumpfiles for different 'selective' dump commands) _Anything_ modified after the last dump at the next lower level needs to be saved as part of -this- dump, unless it is 'excluded' from this dump. A level 0 dump backs up 'everything', except things which are expliclitly excluded. A level 1 dump backs up 'everything that has changed since the last level 0 dump' (whether or not it was inclued in the level 0 dump), except for things which are eplicitly excluded. A subsequent level 1 dump catches everything since the last level _0_ dump. A level 2 dump backs up 'everything that has changed since the last level 1 dump' (whether or not it was inclued in the level 1 dump, or level 0 dump), except for things which are eplicitly excluded. A level 3 dump backs up 'everything that has changed since the last level 2 dump' (whether or not it was inclued in the level 2, 1, or 0 dump), except for things which are eplicitly excluded. A subsequent level 2 dump catches everything since the last level _1_ dump. Another level 1 dump -here- catches everything singe the prior level 0 dump. and kills the record of any level 2 (or higher) dump. A level 3 dump here -- with dump anything since the level 1 dump (since there was no 'current' level 2 dump). So the obvious question is: If I am going to use rsync to keep my backup disk up-to-date, does rsync handle all of these additional small but important filesystem details properly too? If not, then I had best just stick with dump/restore, and use the dumplevels. It doesn't matter what anybody tells you. what matters is what the machine does. Thus the authoritative response is try it and find out. grin I was first asking about in this thread, i.e. will I have successfully created a _bootable_ mirror of my main system disk... With some tuning, yes. /etc/fstab mountpoints often won't match when such a drive is connected to a different port or new system. Right. There's that. So I will need to diddle the /etc/fstab in the backup. 'dump'/'restore' doesn't do -everything- necessary. You have to: set up slices/partitions as needed. 'label' as appropriate INSTALL BOOT BLOCKS 'newfs' the appropriate filesystems then dump/restore the filesystems on the disk. Likewise with the Ethernet board, so see rc.conf(5) about
Re: Making a bootable backup (hard)disk... how?
Ronald F. Guilmette writes: Warren? Just a couple more quick questions. You recommend: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) I'm real curious about you suggestions for the -C and -b values. I have what amounts to a personal workstation. Yea, OK, it is running mail, web, and FTP servers also, but fundementailly, it is not that busy most of the time. And it's got 4GB of main installed. On average, I suspect that I ain't even using half of that. Given all that, why shouldn't I specify (e.g.): -C512 -b1024 ? Wouldn't that all make the dump go faster? There are many possible obstacles to faster dump speed; enumerating them is left as an exercise for the reader. As it happens, I have a set-up very similar to what you describe ... except with 8g of memory. A few years ago I did some testing with various cache sizes (as part of diagnosing other problems) and ended up with C=32. (I should probably run the tests again, given some hardware changes since.) That machine's level 0 runs tonight; I will try to remember to (retain and) post the results. Robert Huff ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Sun, 10 Jun 2012, Robert Huff wrote: Ronald F. Guilmette writes: Warren? Just a couple more quick questions. You recommend: dump -C16 -b64 -0uanL -h0 -f - /usr | (cd /mnt restore -ruf -) I'm real curious about you suggestions for the -C and -b values. I have what amounts to a personal workstation. Yea, OK, it is running mail, web, and FTP servers also, but fundementailly, it is not that busy most of the time. And it's got 4GB of main installed. On average, I suspect that I ain't even using half of that. Given all that, why shouldn't I specify (e.g.): -C512 -b1024 ? Wouldn't that all make the dump go faster? There are many possible obstacles to faster dump speed; enumerating them is left as an exercise for the reader. As it happens, I have a set-up very similar to what you describe ... except with 8g of memory. A few years ago I did some testing with various cache sizes (as part of diagnosing other problems) and ended up with C=32. (I should probably run the tests again, given some hardware changes since.) That machine's level 0 runs tonight; I will try to remember to (retain and) post the results. -C16 is a little small because it's hard to tell how much memory someone running the example will have. I usually use -C32 on machines with plenty of memory. Remember that dump forks multiple times, so it's not 32M, but N*32M. -b64 is the largest safe buffer size to use. People have experienced problems with larger amounts. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On 06/09/12 00:58, Warren Block wrote: On Fri, 8 Jun 2012, Robert Huff wrote: Ronald F. Guilmette writes: I got a lot of disks here, so that part is not a problem. I just need to make sure that I'm gonna do this the Right Way[tm]. (I've already been making my own ham-fisted disk-to-disk backups in the past, but I'm sure that the way I have been doing that is sub-optimal, so I'm here seeking knowledge of how to do this the Right Way.) The bottom line is this... I know how to use cpio, and would like to use it to create a complete and _bootable_ backup of my main system disk. (My main system disk has only one BIOS partition, and that is sub-divided into the usual set of FreeBSD partitions, you know, /, /dev, /tmp, /usr, /var, /usr/compat/linux/proc, and /var/named/dev.) As far as I know, the only way guaranteed to preserve metadata is dump/restore. See previous (not necessarily recent) discussion (on this list, and possibly in the Handbook) for more information. The rsync port has a flags option. I haven't tried it for a full backup. Even if it can copy all filesystem attributes like dump, there are still non-filesystem things needed for booting that neither can copy, like partition tables and boot blocks. There's a BFI (brute force and ignorance) way of doing it in the base system - dd. Provided your system disk is quiescent (ideally when running from a live CD or all partitions mounted read-only, otherwise pray to the deity of your choice) and the backup disk is a) at least as large as the system disk, and b) has the same sector size, then a simple dd from the system disk to the backup should work. [Greybeard war story:] Going back about 25-30 years, a friend of mine was responsible for running the Unix systems in an EE department of a UK university. He used to back up the disks (probably around 10 MB in those days) to 1/2 tape every night. Eventually he got sick of undergraduates asking him to restore files they'd accidentally deleted, and hit upon the idea of dd'ing the disk to tape, and then if a student wanted a restore, he'd mount the tape as a r/o file system (these were the days of tape block devices and trusting all users) and tell them to use restore to get the files themselves. It took forever, but it was the student's time that was wasted, not his. Most of them learned not to delete wanted files after one or two times of doing this. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
In message 4fd38b9a.4010...@qeng-ho.org, Arthur Chance free...@qeng-ho.org wrote: There's a BFI (brute force and ignorance) way of doing it in the base system - dd. Provided your system disk is quiescent (ideally when running from a live CD or all partitions mounted read-only, otherwise pray to the deity of your choice) and the backup disk is a) at least as large as the system disk, and b) has the same sector size, then a simple dd from the system disk to the backup should work. Thank you Arthur, and yes, trying to back up a partition that's currently mounted r/w using dd will almost certainly not produce the desired results. And of course, as you correctly note also, the target partition had best be at least as big as the source (and perhaps even identical in size). I don't care to take own my system to make backups... and don't believe that I should have to do so, and thus, this is one of the reasons why I would prefer to use something like cpio. Also, I don't like backups taking longer than absolutely necessary, and this is why I am specifically _not_ attracted to either the dd solution or to dump/restore, because as I understand it, with either of these methods you end up copying perhaps a metric buttload worth of unallocated free disk space. (I would prefer not to do that. It just seems wasteful... of time, if nothing else.) Also, in one case, one of my partitions has one directory that contains a really massive amount of stuff, and I specifically _don't_ need any of that particular stuff (in that one directory) backed up. So again, I'm looking at tar or cpio or perhaps pax. (Of course cpio is more full-featured than tar, and I don't really know anything about pax, so that leaves me with cpio.) [Greybeard war story:] Going back about 25-30 years, a friend of mine was responsible for running the Unix systems in an EE department of a UK university. He used to back up the disks (probably around 10 MB in those days) to 1/2 tape every night. Eventually he got sick of undergraduates asking him to restore files they'd accidentally deleted, and hit upon the idea of dd'ing the disk to tape, and then if a student wanted a restore, he'd mount the tape as a r/o file system (these were the days of tape block devices and trusting all users) and tell them to use restore to get the files themselves. It took forever, but it was the student's time that was wasted, not his. Most of them learned not to delete wanted files after one or two times of doing this. My own greybeard story: Back in the late 70s I was in college and had just switched major to CS. My college had a mainframe, but also a PDP-11 to which were connected a number of ASR-33s (Who could ever forget THOSE!) Students who asked could get an account on the PDP-11/RSTS-E system. I did, and started experimenting. I remember one day I really pissed off the specific Computer Center staff member who was assigned to the care feeding of the RSTS-E system when her backup tape started whizzing back and forth for no apparent reason. At least it seemed that way to her, since she had only just mounted the tape and _she_ had not issued any console commands yet which would have caused the tape drive to do anything. :-) Needless to say RSTS-E was, apparently, not quite as sophisticated at that time as, say, UNIX, in terms of protecting physical I/O devices from general Joe-Schmo users. I never became friends with that specific Computer Center staff member, but I did end up as the only CS undergraduate who got a job in the campus Computer Center. Sorry. You talking about tapes just reminded of that incident, way back when. (Sigh. The good old daze when men were men, and the bits ran scared!) Moral of the story is that sometimes it actually does pay to be a smartass. Regards, rfg P.S. It really is a Damn Shame[tm] that nobody ever hacked FreeBSD cpio to make it be able to copy (a) the extra file flag/mode bits and/or (b) file ACLs and/or (c) file attributes. I guess I have found a new project for myself... when/if I ever find some free time. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Sat, 9 Jun 2012, Ronald F. Guilmette wrote: Also, I don't like backups taking longer than absolutely necessary, and this is why I am specifically _not_ attracted to either the dd solution or to dump/restore, because as I understand it, with either of these methods you end up copying perhaps a metric buttload worth of unallocated free disk space. No, that is one of the biggest advantages of dump over dd. dump knows UFS, and only copies occupied sectors and needed information. That's why it can be restored file-by-file or to a partition of a different size. Also, in one case, one of my partitions has one directory that contains a really massive amount of stuff, and I specifically _don't_ need any of that particular stuff (in that one directory) backed up. So again, I'm looking at tar or cpio or perhaps pax. (Of course cpio is more full-featured than tar, and I don't really know anything about pax, so that leaves me with cpio.) Directories and files can be skipped with the nodump flag. P.S. It really is a Damn Shame[tm] that nobody ever hacked FreeBSD cpio to make it be able to copy (a) the extra file flag/mode bits and/or (b) file ACLs and/or (c) file attributes. A quick search shows dump should support ACLs. The other stuff is also there. Try it. http://www.wonkity.com/~wblock/docs/html/backup.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Making a bootable backup (hard)disk... how?
I've been lucky. Over about the past 20 years I've never had a hard
disk go bad on me. (Knock on wood.) Of course I _do_ only buy the
better quality ones (with the 5 year warranties), and I'm sure that
has helped. Still, one never knows, and it is best to be prepared.
Primarily however, I am mostly afraid of the fumble-finger scenario...
you know... cd /; rm -fr *
I read a little bit about UFS filesystem snashots, and they could probably
help me deal effectively with fumble-finger incidents, but I still think
that I would prefer to have a whole separate physical disk as my backup.
(I mean what the hell? They are so cheap these days anyway.)
I got a lot of disks here, so that part is not a problem. I just
need to make sure that I'm gonna do this the Right Way[tm]. (I've already
been making my own ham-fisted disk-to-disk backups in the past, but I'm
sure that the way I have been doing that is sub-optimal, so I'm here
seeking knowledge of how to do this the Right Way.)
The bottom line is this... I know how to use cpio, and would like to use
it to create a complete and _bootable_ backup of my main system disk.
(My main system disk has only one BIOS partition, and that is sub-divided
into the usual set of FreeBSD partitions, you know, /, /dev, /tmp, /usr,
/var, /usr/compat/linux/proc, and /var/named/dev.)
I'm not a complete dummy, so I can can easily figure out how to use cpio
to make copies of all of the things in these filesystems to another
physical drive, however, there are a couple of issues that I really don't
know how to deal with properly, to wit:
1) How can one induce cpio to copy over the extra magical FreeBSD-specific
file flags, such as the immutable flag, and so forth? (Naturally, I _do_
want to preserve all of these in my backup.)
2) Of necessity, the drive/partition designators of my backup disk and my
``real'' original system disk will be different. The original is a SATA
drive, designator(s) /dev/ad4s1{a,e,f,d}, but due to the limitations of what
I have on hand, my backup drive will be an older PATA drive, designator(s)
/dev/ad2s1{a,e,f,d}. Obviously, cpio doesn't give a darn about such things,
and so making my backup will not be a problem at all. The problem will,
I suspect, arise when and if I actually have to try to use my backup disk to
recover from a system failure. Then, the identity of the drive that will
actually hold the root partition will be new and different from whatever it
is set to currently on my ``real''/original drive... which will probably
cause the disk not to boot... and also, of course, a lot of the entries in
/etc/fstab will be wrong for the backup drive and will have to be diddled
at some time... either at the time I make my backup, or later, when I go
to actually _use_ the backup disk. I need to know he Right Way to deal
with these (drive designator) problems also.
I guess that I can figure out for myself how to diddle the /etc/fstab file
on the backup drive in an appropriate way (and I can probably even cook up
a trivial script to do this diddling for me routinely, each time I make a
new disk-to-disk backup). So I guess the only think that is still making
me nervous is my dim recollection... leftover from FreBSD releases of long
long ago, e.g. the 6.x series... that there was also something hard-coded
somewhere that told the kernel the identity of the root partition. If that
hard coded thing still exists somewhere (i.e. in 8.3) then I'll need to
make it a point, when making my bootbale backup disk, to find and diddle that
thing too, in addition to diddling the /etc/fstab entries.
So my question is simple: Do I need to diddle anything other than /etc/fstab
in order to allow for the fact that all of the drive designators will be
different for my backup disk, as compared to my original disk? And if so,
where is/are the additional things that will need to be manually diddled?
Any help/advice/insight regarding the above topics will be much appreciated.
Regards,
rfg
P.S. Maybe I just overlooked it, but I really didn't see any option listed
on the man page for cpio(1)... aka bsdcpio(1)... that would permit copying/
preservation of the various file flags discussed in the man pages for
chflags(1) and chflags(2). Is it just me, or can cpio really not copy
these? If so, it seems like one helluva big oversight. (Even if there
is as issue where these additional file flags cannot be preserved, i.e.
in cpio archive files, you know, because of backward archive format com-
patability issues, at the very least there could be / should be a way to
get cpio to copy these things over when the -p (pass) option is being used...
no?
Hummm... come to think of it, I'm also not seeing any way to induce cpio
to copy over things like per-file ACLs and/or per-file file attributes
either.
Is it just me, or does _everyone_ who uses cpio for backups loose all of
this good stuff when making backups?
Color me preplexed
Making a bootable backup (hard)disk... how?
Ronald F. Guilmette writes: I got a lot of disks here, so that part is not a problem. I just need to make sure that I'm gonna do this the Right Way[tm]. (I've already been making my own ham-fisted disk-to-disk backups in the past, but I'm sure that the way I have been doing that is sub-optimal, so I'm here seeking knowledge of how to do this the Right Way.) The bottom line is this... I know how to use cpio, and would like to use it to create a complete and _bootable_ backup of my main system disk. (My main system disk has only one BIOS partition, and that is sub-divided into the usual set of FreeBSD partitions, you know, /, /dev, /tmp, /usr, /var, /usr/compat/linux/proc, and /var/named/dev.) As far as I know, the only way guaranteed to preserve metadata is dump/restore. See previous (not necessarily recent) discussion (on this list, and possibly in the Handbook) for more information. Robert Huff ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Fri, 8 Jun 2012, Robert Huff wrote: Ronald F. Guilmette writes: I got a lot of disks here, so that part is not a problem. I just need to make sure that I'm gonna do this the Right Way[tm]. (I've already been making my own ham-fisted disk-to-disk backups in the past, but I'm sure that the way I have been doing that is sub-optimal, so I'm here seeking knowledge of how to do this the Right Way.) The bottom line is this... I know how to use cpio, and would like to use it to create a complete and _bootable_ backup of my main system disk. (My main system disk has only one BIOS partition, and that is sub-divided into the usual set of FreeBSD partitions, you know, /, /dev, /tmp, /usr, /var, /usr/compat/linux/proc, and /var/named/dev.) As far as I know, the only way guaranteed to preserve metadata is dump/restore. See previous (not necessarily recent) discussion (on this list, and possibly in the Handbook) for more information. The rsync port has a flags option. I haven't tried it for a full backup. Even if it can copy all filesystem attributes like dump, there are still non-filesystem things needed for booting that neither can copy, like partition tables and boot blocks. There might be something for a bootable backup in ports, or it could be done with a script using gpart. Set up the disk with GPT labels to make it relocatable, use dump to copy everything the first time, fix up a few things like fstab and ifconfig_DEFAULT in the same script. Afterwards, rsync may be enough for fast updates. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Making a bootable backup (hard)disk... how?
On Fri, Jun 8, 2012 at 4:58 PM, Warren Block wbl...@wonkity.com wrote: On Fri, 8 Jun 2012, Robert Huff wrote: Ronald F. Guilmette writes: I got a lot of disks here, so that part is not a problem. I just need to make sure that I'm gonna do this the Right Way[tm]. (I've already been making my own ham-fisted disk-to-disk backups in the past, but I'm sure that the way I have been doing that is sub-optimal, so I'm here seeking knowledge of how to do this the Right Way.) The bottom line is this... I know how to use cpio, and would like to use it to create a complete and _bootable_ backup of my main system disk. (My main system disk has only one BIOS partition, and that is sub-divided into the usual set of FreeBSD partitions, you know, /, /dev, /tmp, /usr, /var, /usr/compat/linux/proc, and /var/named/dev.) As far as I know, the only way guaranteed to preserve metadata is dump/restore. See previous (not necessarily recent) discussion (on this list, and possibly in the Handbook) for more information. The rsync port has a flags option. I haven't tried it for a full backup. Even if it can copy all filesystem attributes like dump, there are still non-filesystem things needed for booting that neither can copy, like partition tables and boot blocks. There might be something for a bootable backup in ports, or it could be done with a script using gpart. Set up the disk with GPT labels to make it relocatable, use dump to copy everything the first time, fix up a few things like fstab and ifconfig_DEFAULT in the same script. Afterwards, rsync may be enough for fast updates. I am applying the following technique in Linux ( because I am not using FreeBSD ) : On the same computer : (1) I am disconnecting power cable of the existing HDD . (2) I am connecting a new disk ( by attaching Channel Zero SATA Cable to it ) . (3) I am installing a new operating system on HDD (4) I am installing mc ( Midnight Commander ) . (5) After obtaining a working new HDD , I am connecting the power cable of the previous disk . (6) I am booting the computer . The new operating system is seeing previous HDD . (7) I am mounting previous HDD . (8) By using mc , I am copying required files/directories into new HDD . Mostly I am using KDE4 . Dolphin in KDE4 has a memory leak which is causing disaster for very large list of files because for each file , it is consuming irrecoverable memory loss from 600 bytes to 1000 bytes during copying . (9) I am keeping previous HDD as bootable by disconnecting its power cable because it is not necessary to keep it powered . You can do the following : (1) Install your operating system on back-up HDD . (2) By using a script , copy required files /directories onto back-up HDD . With this technique , you can maintain a revolving back-up HDD units : Create back-up HDD 1 . Create back-up HDD 2 . . . . Create back-up HDD n . When n-th HDD is created , and a new back-up is required switch to HDD 1 . Using rsync you may update only required files / directories . Full copy is not required to update subsequent HDD units because it is very likely that many files are the same . In that form , EACH back-up HDD is bootable when it is required . You may consider HAST for a single HDD . You may use external USB HDD by selecting a robust model . Disconnecting power cable of existing HDD during a new operating system install on another HDD is preventing an accidental loss of data because naming the recognized HDD units is NOT standard and very unreliable . Thank you very much . Mehmet Erol Sanliturk ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Backup strategy for zfs + jail
Hi, I have a simple 1U server with two disks that I have configured as a jail server. I want to setup a simple yet very efficient backup policy for my jail environment. This server is running a ZFS filesystem. Ideally I would like to backup the main zfsroot/jail and all subdirectories on a backup FTP server. • What kind of tool would you suggest ? I need to focus on : -- Simplicity of setup -- Ease of recovery -- Efficiency -- Compatibility with ZFS Thanks. –– - Grégory Bernard Director - --- www.osnet.eu --- -- Your provider of OpenSource appliances -- –– OSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetO ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: Backup strategy for zfs + jail
-Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd- questi...@freebsd.org] On Behalf Of bsd Sent: Tuesday, January 17, 2012 1:04 PM To: Liste FreeBSD Subject: Backup strategy for zfs + jail Hi, I have a simple 1U server with two disks that I have configured as a jail server. I want to setup a simple yet very efficient backup policy for my jail environment. This server is running a ZFS filesystem. Ideally I would like to backup the main zfsroot/jail and all subdirectories on a backup FTP server. . What kind of tool would you suggest ? I need to focus on : -- Simplicity of setup -- Ease of recovery -- Efficiency -- Compatibility with ZFS If you're running 9, give HAST a shot. TCP/IP block-level mirroring provided by HAST should be able to mirror the ZFS container in near-RT and be tolerant of things like network issues. -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup strategy for zfs + jail
On 17-01-2012, Tue [13:52:48], Devin Teske wrote: -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd- questi...@freebsd.org] On Behalf Of bsd Sent: Tuesday, January 17, 2012 1:04 PM To: Liste FreeBSD Subject: Backup strategy for zfs + jail Hi, I have a simple 1U server with two disks that I have configured as a jail server. I want to setup a simple yet very efficient backup policy for my jail environment. This server is running a ZFS filesystem. Ideally I would like to backup the main zfsroot/jail and all subdirectories on a backup FTP server. . What kind of tool would you suggest ? I need to focus on : -- Simplicity of setup -- Ease of recovery -- Efficiency -- Compatibility with ZFS If you're running 9, give HAST a shot. TCP/IP block-level mirroring provided by HAST should be able to mirror the ZFS container in near-RT and be tolerant of things like network issues. -- Devin I personally wouldn't rely on such a new technology as HAST, considering the importancy of backups. ZFS has some nice features already. Create snapshots of your datasets and use zfs send. You can even transfer differences between snapshots. Google for it. ZFS is awesome modern technology, more than that it's stable enough. ;) -- Best wishes, Dmitry Sarkisov --\ ---+-- --/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: Backup strategy for zfs + jail
-- Efficiency -- Compatibility with ZFS If you're running 9, give HAST a shot. maybe a stupid question but what is a practical difference between hast and doing ggate+gmirror and setting prefer load balancing to local disk? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup strategy for zfs + jail
Create snapshots of your datasets and use zfs send. You can even transfer differences between snapshots. and then try to recover data from these backups after a year or so ;) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup strategy for zfs + jail
Le 17 janv. 2012 à 22:52, Devin Teske a écrit : -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd- questi...@freebsd.org] On Behalf Of bsd Sent: Tuesday, January 17, 2012 1:04 PM To: Liste FreeBSD Subject: Backup strategy for zfs + jail Hi, I have a simple 1U server with two disks that I have configured as a jail server. I want to setup a simple yet very efficient backup policy for my jail environment. This server is running a ZFS filesystem. Ideally I would like to backup the main zfsroot/jail and all subdirectories on a backup FTP server. . What kind of tool would you suggest ? I need to focus on : -- Simplicity of setup -- Ease of recovery -- Efficiency -- Compatibility with ZFS If you're running 9, give HAST a shot. TCP/IP block-level mirroring provided by HAST should be able to mirror the ZFS container in near-RT and be tolerant of things like network issues. Though HAST seems quite interesting I am not looking for a cluster solution. My FTP server is provided by my ISP, and Ideally I would like something that feets in that solution… So something like ZFS snapshots + some syncing level FTP service that will do the copy to the remote location… I don't really know if this approach will work with a ZFS snapshot, I use to work at file level using duplicity… works well though a bit heavy ! G.B. -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. –– - Grégory Bernard Director - --- www.osnet.eu --- -- Your provider of OpenSource appliances -- –– OSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetO ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup strategy for zfs + jail
On 17-01-2012, Tue [23:31:30], Wojciech Puchar wrote: Create snapshots of your datasets and use zfs send. You can even transfer differences between snapshots. and then try to recover data from these backups after a year or so ;) No one did mention the retention policy ;) Jokes aside, we have a working solution with zfs/symantec netbackup combo based on incremental snapshots for a pretty large datasets. To OP: you don't have to use ftp with zfs send/recieve (I doubt it is possible at all :) ), ssh suits better. Just _google_ for it. There are plenty of solutions/examples in the Net. -- Best wishes, Dmitry Sarkisov --\ ---+-- --/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Need to Backup Using Dump
I have two FreeBSD 8.2-STABLE servers running NFS. I have tons of files on Server A that I want to backup to a big drive on Sever B. Server B nfs_mounts one of the filesystems on Server A to /mnt. So if I wanted to make a backup of the filesytem on Server A to Server B I tried: dump -d /home/my_home/backups/20111024 /mnt but each time I try this it tells me that filesystem /mnt is unknown. /mnt is not in /etc/fstab. I manually mounted this via NFS and that's where all the files I want to backup are accessible to the command line on Server B. What am I missing? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Need to Backup Using Dump
El día Sunday, October 23, 2011 a las 06:04:14AM -0700, Bill Tillman escribió: I have two FreeBSD 8.2-STABLE servers running NFS. I have tons of files on Server A that I want to backup to a big drive on Sever B. Server B nfs_mounts one of the filesystems on Server A to /mnt. So if I wanted to make a backup of the filesytem on Server A to Server B I tried: dump -d /home/my_home/backups/20111024 /mnt but each time I try this it tells me that filesystem /mnt is unknown. /mnt is not in /etc/fstab. I manually mounted this via NFS and that's where all the files I want to backup are accessible to the command line on Server B. What am I missing? Following dump(8) the file system to be dumped must be a device special file or its mountpoint from fstab(5). HIH matthias -- Matthias Apitz t +49-89-61308 351 - f +49-89-61308 399 - m +49-170-4527211 e g...@unixarea.de - w http://www.unixarea.de/ 200†-20††: 10 años de guerra en Afghanistan. ¡Basta ya! 200†-20††: 10 years war in Afghanistan. Stop it now! 200†-20††: 10 Jahre Krieg in Afghanistan. Schluss jetzt endlich! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Need to Backup Using Dump
On Sun, 23 Oct 2011, Bill Tillman wrote: I have two FreeBSD 8.2-STABLE servers running NFS. I have tons of files on Server A that I want to backup to a big drive on Sever B. Server B nfs_mounts one of the filesystems on Server A to /mnt. So if I wanted to make a backup of the filesytem on Server A to Server B I tried: dump -d /home/my_home/backups/20111024 /mnt but each time I try this it tells me that filesystem /mnt is unknown. /mnt is not in /etc/fstab. I manually mounted this via NFS and that's where all the files I want to backup are accessible to the command line on Server B. What am I missing? dump(8) works on entire filesystems. To copy directories, consider rsync. To copy directories but put them into archives, there's tar and numerous others. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Need to Backup Using Dump
On Sun, 23 Oct 2011 06:04:14 -0700 (PDT), Bill Tillman wrote: I have two FreeBSD 8.2-STABLE servers running NFS. I have tons of files on Server A that I want to backup to a big drive on Sever B. Server B nfs_mounts one of the filesystems on Server A to /mnt. So if I wanted to make a backup of the filesytem on Server A to Server B I tried: dump -d /home/my_home/backups/20111024 /mnt but each time I try this it tells me that filesystem /mnt is unknown. /mnt is not in /etc/fstab. I manually mounted this via NFS and that's where all the files I want to backup are accessible to the command line on Server B. What am I missing? The dump + restore mechanism operates on device files representing a file system, not a _mounted_ file system, as source. If, for example, your /home partition is /dev/ad0s1e, and you've mounted the target at /mnt, then you could do: # cd /mnt # dump -0 -f - /dev/ad0s1e | restore -r -f - Therefore /home has to be unmounted (or add flag -L to dump from mounted file systems). But note that this will dump the _complete_ file system's content to /mnt as dump cannot be more selective here. An alternative is to use rsync or cpdup where you can explicitely address a subtree to be copied instead of the whole file system. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Need to Backup Using Dump
Hello. 2011/10/23 21:08:14 +0200 Polytropon free...@edvax.de = To Bill Tillman : P The dump + restore mechanism operates on device files P representing a file system, not a _mounted_ file system, P as source. dump(8) is able to make a snapshot behind teh scenes, and use that snapshot as a source for dump: -L This option is to notify dump that it is dumping a live file sys tem. To obtain a consistent dump image, dump takes a snapshot of the file system in the .snap directory in the root of the file system being dumped and then does a dump of the snapshot. -- Peter Vereshagin pe...@vereshagin.org (http://vereshagin.org) pgp: A0E26627 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Need to Backup Using Dump
On Sun, 23 Oct 2011, Polytropon wrote: But note that this will dump the _complete_ file system's content to /mnt as dump cannot be more selective here. The nodump flag along with -h0 can be used to exclude files or directories. But agreed, dump is gear towards entire filesystems. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
ZFS on root backup
OK, so now my ZFS on root FreeBSD-8.2 system runs smoothly and I'm very happy being able to have ZFS (coming from solaris11), but.. what is the best strategy to back this fbsd system up. do I create various ZFS backup filesystem streams or can I easely backup the zroot pool as a whole? And if yes, how? Grateful for all the help I can get in these matters. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS on root backup
On 11/07/2011 11:20, Dick Hoogendijk wrote: OK, so now my ZFS on root FreeBSD-8.2 system runs smoothly and I'm very happy being able to have ZFS (coming from solaris11), but.. what is the best strategy to back this fbsd system up. do I create various ZFS backup filesystem streams or can I easely backup the zroot pool as a whole? And if yes, how? Grateful for all the help I can get in these matters. Well, the best backup strategy is a very site specific thing. There's no one-size-fits-all answer. Suppose your machine caught fire right now. You're always going to lose some data -- everything that changed since the last backup. How much data can you afford to lose like that? A day's worth? A week's worth? Absolutely none at all? Given your server is destroyed, how fast do you need to get it back? Can you wait for a week in order to source replacement kit? Or does every second of lost uptime cost you significant amounts of money? Now imagine the ultimate disaster scenario: typically something like -- a fully loaded jumbo jet crashes into your datacenter and everything is completely destroyed[*]. Will your backups survive? Can you rebuild your system using those backups? Suppose someone stole your backup media: can they extract all your data from them, and does it matter if they do? Those are the sorts of questions you need to think about when designing a backup system. There's nothing particularly specific about ZFS there. What ZFS does get you in terms of backups are two things: 1) Really easy and unlimited amounts of snap-shotting. As well as making it really simple to get a coherent point-in-time backup of an active filesystem, they also give you a really simple 'undo' type functionality, so you can unwind accidental deletions and other user mistakes. 2) ZFS import and export -- again, exploiting the snap-shotting capability, this makes it pretty easy to create a duplicate of your filesystem onto another host, and to update the duplicate in a very efficient way. Cheers, Matthew [*] As a number of companies found out to their cost, 'in the basement of the other tower' was not sufficiently off-site to be effective. -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matt...@infracaninophile.co.uk Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: ZFS on root backup
On 7/11/2011 5:59 AM, Matthew Seaman wrote: On 11/07/2011 11:20, Dick Hoogendijk wrote: OK, so now my ZFS on root FreeBSD-8.2 system runs smoothly and I'm very happy being able to have ZFS (coming from solaris11), but.. what is the best strategy to back this fbsd system up. do I create various ZFS backup filesystem streams or can I easely backup the zroot pool as a whole? And if yes, how? Grateful for all the help I can get in these matters. [...] What ZFS does get you in terms of backups are two things: 1) Really easy and unlimited amounts of snap-shotting. As well as making it really simple to get a coherent point-in-time backup of an active filesystem, they also give you a really simple 'undo' type functionality, so you can unwind accidental deletions and other user mistakes. 2) ZFS import and export -- again, exploiting the snap-shotting capability, this makes it pretty easy to create a duplicate of your filesystem onto another host, and to update the duplicate in a very efficient way. Cheers, Matthew [*] As a number of companies found out to their cost, 'in the basement of the other tower' was not sufficiently off-site to be effective. I recently just converted my system from raid0 to raid1, using zfs, and because of mountpoints and other quirks you could end up having trouble with zfs send and receive. If you're planning on backing up the whole pool, then using a backup drive, and virtualbox with the disk image on the backup drive to backup the pool might be easiest after you get networking working. If you're not concerned about the whole pool, just just create filesystems for what you want to save and use those. You could just use the zfs send command and not recreate the filesystem, but it's not recommended until if and when the data stream is standardized. It'll work until Oracle breaks it. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS on root backup
Matthew == Matthew Seaman m.sea...@infracaninophile.co.uk writes: Matthew1) Really easy and unlimited amounts of snap-shotting. As well as Matthew making it really simple to get a coherent point-in-time backup of Matthew an active filesystem, they also give you a really simple 'undo' Matthew type functionality, so you can unwind accidental deletions and Matthew other user mistakes. And with zxfer (in ports), very simple means to transfer those snapshots to another location. I'm using that now on multiple machines, and it's nearly flawless. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 mer...@stonehenge.com URL:http://www.stonehenge.com/merlyn/ Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc. See http://methodsandmessages.posterous.com/ for Smalltalk discussion ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
gptboot: invalid GPT backup header on soekris net5501
Hello, I formatted a ssd with the following procedure: gpart create -s gpt ad0 gpart add -t freebsd-boot -l sboot -s 64K ad0 gpart bootcode -b /mnt/boot/pmbr -p /mnt/boot/gptboot -i 1 ad0 gpart add -t freebsd-ufs -l sroot -b 2048 -s 1G ad0 gpart add -t freebsd-ufs -l svar -s 1G ad0 gpart add -t freebsd-ufs -l susr ad0 newfs -b 32768 -f 4096 -L root /dev/ad0p2 newfs -U -b 32768 -f 4096 -L var /dev/ad0p3 newfs -U -b 32768 -f 4096 -L usr /dev/ad0p4 After installing FreeBSD 8.2 the system (a soekris net5501) hangs on boot with gptboot: invalid GPT backup header . I'm not sure if this is a hardware problem or if I make a mistake during the setup. What is the cause of the error message? Many thanks Matthias ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Tape drive for backup soloution
Hi I need to implement a tape drive backup solution at my place of work. I was wondering what is a good tape drive to get for this task, that works on freebsd with something like amanda. Its for a small business, and storing about 4TB max, and hopefully with some room spare for differential backups over time. My server on hand has sata and IDE available. Are there any recommendations that you can make about compatible solutions. My knowledge in this area is limited. Sincerely, William Brown Research Teaching, Technology Services The University of Adelaide, AUSTRALIA 5005 CRICOS Provider Number 00123M - IMPORTANT: This message may contain confidential or legally privileged information. If you think it was sent to you by mistake, please delete all copies and advise the sender. For the purposes of the SPAM Act 2003, this email is authorised by The University of Adelaide. pgp.mit.edu PGP.sig Description: This is a digitally signed message part
Re: Tape drive for backup soloution
On Tue, 29 Mar 2011 16:56:18 +1030 William Brown william.e.br...@adelaide.edu.au wrote: Are there any recommendations that you can make about compatible solutions. My knowledge in this area is limited. My needs are not so big and I use HP Ultrium 448 (LTO-2) drive, but I'm sure that buying HP's LTO-2 drive will be nice solution for you. btw, I recently switched from Bacula to Amanda, but I'm just in the process of moving to (Free)PCBSD, but I'm sure drive is supported well. Otoh, I also heard that IBM's drives are not bad and usually cheaper than HP brand. Sincerely, Gour -- “In the material world, conceptions of good and bad are all mental speculations…” (Sri Caitanya Mahaprabhu) http://atmarama.net | Hlapicina (Croatia) | GPG: CDBF17CA signature.asc Description: PGP signature
Half a Mirror Backup
Is using one half of a mirror as a backup a good/bad idea? I was thinking of rotating drives on a periodic basis as a back up method. You'd get the backup instantly, but rebuilding the mirror with the incoming drive would take a little time and leave you vulnerable to a small loss of data if a disk failed while the mirror was rebuilding. Regards, Jason ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Half a Mirror Backup
On Sun, Jan 2, 2011 at 11:56 AM, Jason C. Wells j...@speakeasy.net wrote: Is using one half of a mirror as a backup a good/bad idea? I was thinking of rotating drives on a periodic basis as a back up method. You'd get the backup instantly, but rebuilding the mirror with the incoming drive would take a little time and leave you vulnerable to a small loss of data if a disk failed while the mirror was rebuilding. Besides the problem you mention, you'll have a pretty sizable performance hit while the mirror is being rebuilt. Also, keep in mind that the most likely time for a second drive to fail is during a rebuild, since the rebuild forces a read from every sector. I think I would use rsync or dump instead, although I have to admit the rotating mirror idea is clever. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Half a Mirror Backup
David Brodbeck wrote: On Sun, Jan 2, 2011 at 11:56 AM, Jason C. Wells j...@speakeasy.net wrote: Is using one half of a mirror as a backup a good/bad idea? I was thinking of rotating drives on a periodic basis as a back up method. You'd get the backup instantly, but rebuilding the mirror with the incoming drive would take a little time and leave you vulnerable to a small loss of data if a disk failed while the mirror was rebuilding. Besides the problem you mention, you'll have a pretty sizable performance hit while the mirror is being rebuilt. Also, keep in mind that the most likely time for a second drive to fail is during a rebuild, since the rebuild forces a read from every sector. I think I would use rsync or dump instead, although I have to admit the rotating mirror idea is clever. Ref rsync: Personally I use rdist6, (from familiarity=habit rather than conscious choice, (it's more limited predecessor rdist, has been in BSD a Long time)) But beware: rdist6 fails on files bigger than 2G on i686 but (not on amd64, on amd64 no problem), I wouldnt know if rsync might have a similar 2G restriction. Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, or HTML or base 64. Avoid top posting, it cripples itemised cumulative responses. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Half a Mirror Backup
On 02/01/2011 20:30, David Brodbeck wrote:
On Sun, Jan 2, 2011 at 11:56 AM, Jason C. Wells j...@speakeasy.net wrote:
Is using one half of a mirror as a backup a good/bad idea?
I was thinking of rotating drives on a periodic basis as a back up method.
You'd get the backup instantly, but rebuilding the mirror with the incoming
drive would take a little time and leave you vulnerable to a small loss of
data if a disk failed while the mirror was rebuilding.
Besides the problem you mention, you'll have a pretty sizable
performance hit while the mirror is being rebuilt. Also, keep in mind
that the most likely time for a second drive to fail is during a
rebuild, since the rebuild forces a read from every sector. I think I
would use rsync or dump instead, although I have to admit the rotating
mirror idea is clever.
It's a well known and really pretty clever technique to obtain a
point-in-time image of a filesystem without needing lengthy outages.
Two points:
1) The classic way of doing this without losing resilience is to use
a mirror of three disks. Typically you'ld attach a third drive
to the existing mirror, let it synch its contents, detach it and
then backup the third disk's content to tape at leisure.
2) It's rendered pretty much obsolete if you have a filesystem that
you can snapshot. Which you can for both UFS and ZFS. Take
snapshots of your filesystems, mount the snapshots and back them
up. It's exceedingly easy with ZFS(*), and only slightly more
involved with UFS.
In fact, snapshotting ZFS is so easy, that it is common practice to
create snapshots at regular intervals -- hourly is not unknown -- and
keep several days worth on-line as insurance against accidental
deletions and so forth.
Note: this does *not* constitute a proper backup. Without off-line
copies (or preferably off-site copies) if your computer gets destroyed
by fire, flood or whatever, then you will loose your data.
Cheers,
Matthew
(*) Hint: ZFS snapshots automount at
/${filesystem}/.zfs/snapshot/${snapname} if you just cd to that directory.
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk Kent, CT11 9PW
signature.asc
Description: OpenPGP digital signature
ultimate backup choice
i have: SERVER A SERVER B with full root permisson [ssh, etc] each server has a folder. i want to backup a folder in SERVER A. are there any backup methods, that meets these two requirements? : 1) running from e.g.: a cronjob 2) when running, it just checks the folder in SERVER A and SERVER B. if a file/folder has been added/removed/modified in the SERVER A's folder, then it copies/removes it/them to SERVER B's folder. please help! Thank you in advance! [for any links, howtos :\ ] have a nice weekend ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ultimate backup choice
On 11 dec 2010, at 13:17, S Mathias wrote: i have: SERVER A SERVER B with full root permisson [ssh, etc] each server has a folder. i want to backup a folder in SERVER A. are there any backup methods, that meets these two requirements? : 1) running from e.g.: a cronjob 2) when running, it just checks the folder in SERVER A and SERVER B. if a file/folder has been added/removed/modified in the SERVER A's folder, then it copies/removes it/them to SERVER B's folder. please help! Thank you in advance! [for any links, howtos :\ ] http://everythinglinux.org/rsync/ -- Peter Boosten http://www.boosten.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ultimate backup choice
On Sat, 11 Dec 2010 04:17:46 -0800 (PST), S Mathias smathias1...@yahoo.com wrote: each server has a folder. Allow me a polite sidenote: Please try to use the correct terminology. Those are called directories, not folders. i want to backup a folder in SERVER A. are there any backup methods, that meets these two requirements? : 1) running from e.g.: a cronjob 2) when running, it just checks the folder in SERVER A and SERVER B. if a file/folder has been added/removed/ modified in the SERVER A's folder, then it copies/removes it/them to SERVER B's folder. Maybe cpdup can help here. Anyway, rsync would be a recommended choice. After installing them, you'll dinf documentation in man cpdup and man rsync, /usr/local/share/doc/rsync/ - those are well documented. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ultimate backup choice
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 1) running from e.g.: a cronjob 2) when running, it just checks the folder in SERVER A and SERVER B. if a file/folder has been added/removed/ modified in the SERVER A's folder, then it copies/removes it/them to SERVER B's folder. http://www.freebsd.org/doc/handbook/backup-basics.html William Brown pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) iQIcBAEBAgAGBQJNA6QxAAoJEHF16AnLoz6JRzsP/2rtrtu7a1vPc3jmhOMppg9q YJYcFTAOcVDR6rjb+82WBn75GgkhGWGfVmOwUThbyu/5kzfvsb70gVzWdia6RcuG Dr5XBFjObYQYRBme7sJBRuRZH1xwb3fzFzbkBf/moi2GvH7yi98FKAkqU8jtmgYN +/Qls+BHYthE8We7HV+02FGmC1s0uC6boebjdrwSFLdBjvM5ehUg4Lb7UL40bckz piC6OpL4vkRmx3VtpsxShvunljaCctiOAkYrs77TMIkooa7pV1kVBbvadUCylx/1 /YZpuz33KnL5+Sd+69HDNOkd+4Ix0a/SPglZdVif9qPt8yAFraxMxqCDauddzEyX bzp6qlEMoSxW8oLptImkI91dYjoMJlT5V51rF5UN50jVlfMKGp4JydK2/T9WBYY3 zN1KagMtzmhz0wgN+Fy2TEh7K034IDtVLZVu34ZGPVgJh8C3+b41CKD2JdtQSMXD 2R64rSerJw3Di9b9Gm/Djt27TR32HDagj/UUH24Gn4uArOkkccmYYgpVR2BrQXPf D6WlopScMhzpRGa7U1IvLMdfe7gLLqVWFQmCkd/mTz2RhT5gBiB2kSOwVtePGDnC dKeM9ahBo+JNC8wSqZq64KOPoqqAkWNRB0oQxjcsP+IsTx8Q+vNDNebjENSz2sBo woXN30B9tPU3Ea21Zqwx =GHAI -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ultimate backup choice
On 11 December 2010 16:17, Indexer inde...@internode.on.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 1) running from e.g.: a cronjob 2) when running, it just checks the folder in SERVER A and SERVER B. if a file/folder has been added/removed/ modified in the SERVER A's folder, then it copies/removes it/them to SERVER B's folder. http://www.freebsd.org/doc/handbook/backup-basics.html William Brown pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) iQIcBAEBAgAGBQJNA6QxAAoJEHF16AnLoz6JRzsP/2rtrtu7a1vPc3jmhOMppg9q YJYcFTAOcVDR6rjb+82WBn75GgkhGWGfVmOwUThbyu/5kzfvsb70gVzWdia6RcuG Dr5XBFjObYQYRBme7sJBRuRZH1xwb3fzFzbkBf/moi2GvH7yi98FKAkqU8jtmgYN +/Qls+BHYthE8We7HV+02FGmC1s0uC6boebjdrwSFLdBjvM5ehUg4Lb7UL40bckz piC6OpL4vkRmx3VtpsxShvunljaCctiOAkYrs77TMIkooa7pV1kVBbvadUCylx/1 /YZpuz33KnL5+Sd+69HDNOkd+4Ix0a/SPglZdVif9qPt8yAFraxMxqCDauddzEyX bzp6qlEMoSxW8oLptImkI91dYjoMJlT5V51rF5UN50jVlfMKGp4JydK2/T9WBYY3 zN1KagMtzmhz0wgN+Fy2TEh7K034IDtVLZVu34ZGPVgJh8C3+b41CKD2JdtQSMXD 2R64rSerJw3Di9b9Gm/Djt27TR32HDagj/UUH24Gn4uArOkkccmYYgpVR2BrQXPf D6WlopScMhzpRGa7U1IvLMdfe7gLLqVWFQmCkd/mTz2RhT5gBiB2kSOwVtePGDnC dKeM9ahBo+JNC8wSqZq64KOPoqqAkWNRB0oQxjcsP+IsTx8Q+vNDNebjENSz2sBo woXN30B9tPU3Ea21Zqwx =GHAI -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org if you are zfs and the said dirs are zfs file systems you could do incremental zfs send and receives between each box ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
