Re: ZFS backups: retrieving a few files?
Quoting Peter Jeremy peterjer...@acm.org (from Wed, 24 Nov 2010 06:32:07 +1100): BTW, the entire export is performed at the current compression level - recompressing existing data. Are you sure the compression is done on the sending side, and not at the receiving side? I would expect the later (as I can specify a different compression level on an existing destination, if I remember correctly). Bye, Alexander. -- Kansas state law requires pedestrians crossing the highways at night to wear tail lights. http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 2010-Nov-24 11:07:23 +0100, Alexander Leidinger alexan...@leidinger.net wrote: Quoting Peter Jeremy peterjer...@acm.org (from Wed, 24 Nov 2010 06:32:07 +1100): BTW, the entire export is performed at the current compression level - recompressing existing data. Are you sure the compression is done on the sending side, and not at the receiving side? I would expect the later (as I can specify a different compression level on an existing destination, if I remember correctly). Sorry, that was poorly worded. The actual send stream is not compressed but the entire filesystem stream will be re-compressed on the receive side as specified by the compression parameter on the sending filesystem. -- Peter Jeremy pgpiHfu1rdR5j.pgp Description: PGP signature
Re: ZFS backups: retrieving a few files?
On Mon, Nov 22, 2010 at 10:26:30AM -0500, Jonathan Stewart wrote: On 11/22/2010 6:35 AM, Andrew Reilly wrote: Dump/restore doesn't work for ZFS. I *think* that I'm running backups in the appropriate equivalent fashion: I take file system snapshots (both absolute == level 0) and relative (incremental), and zfs send those to files on the backup disk. This is actively discouraged, there is no recovery ability when receiving zfs streams so 1 bad bit would invalidate your entire backup. The currently accepted practice is to create a ZFS file system on the backup drive and just keep sending incremental snapshots to it. As long as the backup drive and host system have a snapshot in common you can do incremental transfers. This way you only have to keep the most recent snapshot on the main system and can keep as many as you have space for on the backup drive. You also have direct access to any backed up version of every file. For those playing along at home, I'll issue a small warning, based on today's frolics: Say, for example, one had done a: zfs send -vR tank/h...@0 | zfs receive -d /backup/snapshots in order to experiment with this strategy. One would then become alarmed when one discovered that the receive mechanism also invoked the mountpoint= parameter of the source filesystem, and the zfs propensity for just doing stuff, and boom: you have a read-only version of your home directory mounted *on top of* your actual home directory... Required a reboot to single user mode, to go in and reset the mountpoint setting for the newly created file system (by way of hitting the power switch, after using zfs unmount -f to royally screw things up, preventing subsequent network logins.) Left wondering how to manage that change as part of an automated backup schedule. I think that this backup strategy has a few sharp edges... No, I don't like tar, rsync and friends for backups: they don't deal well with hard links, special files or sparse files. Cheers, -- Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 11/23/10 1:45 PM, Andrew Reilly wrote: No, I don't like tar, rsync and friends for backups: they don't deal well with hard links, special files or sparse files. rsync -avHxS --delete --numeric-ids /src/. /dst/. Handles sparse files (S) and hard links (H). Never had any trouble with special files. What sort of special files are not handled correctly by rsync? I'd like to know because I'm relying on rsync for backups for years on my home network. Thomas ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On Tue, Nov 23, 2010 at 4:56 AM, Thomas Ronner tho...@ronner.org wrote: On 11/23/10 1:45 PM, Andrew Reilly wrote: No, I don't like tar, rsync and friends for backups: they don't deal well with hard links, special files or sparse files. rsync -avHxS --delete --numeric-ids /src/. /dst/. Handles sparse files (S) and hard links (H). Never had any trouble with special files. What sort of special files are not handled correctly by rsync? I'd like to know because I'm relying on rsync for backups for years on my home network. One problem with using rsync when dealing with hard-linked files: it doesn't like it when the source switches from hard-linked to non-hard-linked files. You end up with a mix of hard-linked and non-hard-linked files in the destination, with the contents of the non-hard-linked files all mixed around. We just discovered this when we upgraded our Debian 4.0 (Etch) boxes to Debian 5.0 (Lenny). On Etch, all the gzip tools are hard-links (zcat, zless, gzip, gunzip, etc). On Lenny, they are all separate files, and most are just shell scripts that use gzip. Doing an rsync of a Lenny box onto a directory from an Etch box, you end up with some hard-linked files, some regular files, and the contents of all the files are mixed-up based on which source file (script or binary) was read first. We've had to resort to clearing out the backups directory when doing a Debian upgrade, in order to guarantee that we get a clean backup via rsync. -- Freddie Cash fjwc...@gmail.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 11/23/10 4:45 PM, Freddie Cash wrote: On Tue, Nov 23, 2010 at 4:56 AM, Thomas Ronnertho...@ronner.org wrote: rsync -avHxS --delete --numeric-ids /src/. /dst/. One problem with using rsync when dealing with hard-linked files: it doesn't like it when the source switches from hard-linked to non-hard-linked files. You end up with a mix of hard-linked and non-hard-linked files in the destination, with the contents of the non-hard-linked files all mixed around. We just discovered this when we upgraded our Debian 4.0 (Etch) boxes to Debian 5.0 (Lenny). On Etch, all the gzip tools are hard-links (zcat, zless, gzip, gunzip, etc). On Lenny, they are all separate files, and most are just shell scripts that use gzip. Doing an rsync of a Lenny box onto a directory from an Etch box, you end up with some hard-linked files, some regular files, and the contents of all the files are mixed-up based on which source file (script or binary) was read first. We've had to resort to clearing out the backups directory when doing a Debian upgrade, in order to guarantee that we get a clean backup via rsync. Thanks for this info! I'm going to try to reproduce this. Thomas. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
RE: ZFS backups: retrieving a few files?
If you end up in the double-mounted situation mentioned below, use mount -v to find the fsid for each double-mounted filesystem and then umount them using the fsid. To avoid this, always use the -R option (temporary, alternate root mount point) with zpool if you are working with a root pool. This also applies if you boot off of a CD or DVD and want to examine or modify a root pool. FWIW, I have successfully tested a bare metal restore from ZFS full and incremental send/receive backups (from an NFS mount off of a DataDomain). So, risks aside, it does work. I believe gzipped tar files would also fail if a bit flipped. -Original Message- From: owner-freebsd-sta...@freebsd.org [mailto:owner-freebsd-sta...@freebsd.org] On Behalf Of Andrew Reilly Sent: Tuesday, November 23, 2010 4:46 AM To: Jonathan Stewart Cc: sta...@freebsd.org Subject: Re: ZFS backups: retrieving a few files? On Mon, Nov 22, 2010 at 10:26:30AM -0500, Jonathan Stewart wrote: On 11/22/2010 6:35 AM, Andrew Reilly wrote: Dump/restore doesn't work for ZFS. I *think* that I'm running backups in the appropriate equivalent fashion: I take file system snapshots (both absolute == level 0) and relative (incremental), and zfs send those to files on the backup disk. This is actively discouraged, there is no recovery ability when receiving zfs streams so 1 bad bit would invalidate your entire backup. The currently accepted practice is to create a ZFS file system on the backup drive and just keep sending incremental snapshots to it. As long as the backup drive and host system have a snapshot in common you can do incremental transfers. This way you only have to keep the most recent snapshot on the main system and can keep as many as you have space for on the backup drive. You also have direct access to any backed up version of every file. For those playing along at home, I'll issue a small warning, based on today's frolics: Say, for example, one had done a: zfs send -vR tank/h...@0 | zfs receive -d /backup/snapshots in order to experiment with this strategy. One would then become alarmed when one discovered that the receive mechanism also invoked the mountpoint= parameter of the source filesystem, and the zfs propensity for just doing stuff, and boom: you have a read-only version of your home directory mounted *on top of* your actual home directory... Required a reboot to single user mode, to go in and reset the mountpoint setting for the newly created file system (by way of hitting the power switch, after using zfs unmount -f to royally screw things up, preventing subsequent network logins.) Left wondering how to manage that change as part of an automated backup schedule. I think that this backup strategy has a few sharp edges... No, I don't like tar, rsync and friends for backups: they don't deal well with hard links, special files or sparse files. Cheers, -- Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 11/22/2010 8:29 PM, David Magda wrote: On Nov 22, 2010, at 17:13, Andrew Reilly wrote: The currently accepted practice is to create a ZFS file system on the backup drive and just keep sending incremental snapshots to it. As long as the backup drive and host system have a snapshot in common you can do incremental transfers. This way you only have to keep the most recent snapshot on the main system and can keep as many as you have space for on the backup drive. You also have direct access to any backed up version of every file. That sounds like a very cool notion. Not unlike the time-machine scheme. Interesting how different capabilities require going back and re-thinking the problem, rather than just trying to implement the old solution with the new tools. As noted, saving the output of zfs send isn't very useful and generally not recommended as a backup mechanism. It's come up quite often on Sun/Oracle's zfs-discuss list: http://www.google.com/search?q=zfs+send/receive+as+backup+tool In addition to regular snapshots, also make sure to have an offline backup of some kind (tar, Networker, NetBackup, Amanada, etc.). Errors can propagate to online copies / backups, and if an intruder can penetrate your primary system, there's a good chance they can get to the secondary copy of your data; penetrating a tape on a shelf over the network would be much more challenging. :) I am still trying to figure out the best way to do zfs backups locally here for rollbacks as well as DR. I was looking at some of the techniques at http://www.cuddletech.com/blog/pivot/entry.php?id=984 But thats outdated ? WRT errors in the file, perhaps PAR* tools can overcome some of these issues if you are dumping to a file on tape */usr/ports/archivers/par2cmdline ---Mike ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On Tue, November 23, 2010 14:14, Mike Tancsa wrote: I am still trying to figure out the best way to do zfs backups locally here for rollbacks as well as DR. I was looking at some of the techniques at http://www.cuddletech.com/blog/pivot/entry.php?id=984 But thats outdated ? WRT errors in the file, perhaps PAR* tools can overcome some of these issues if you are dumping to a file on tape */usr/ports/archivers/par2cmdline The above is still quite valid, and snapshots are one of base principles of ZFS. From the official ZFS Admin Guide: The following solutions for saving ZFS data are provided: * Saving ZFS snapshots and rolling back snapshots, if necessary. * Saving full and incremental copies of ZFS snapshots and restoring the snapshots and file systems, if necessary. * Remotely replicating ZFS file systems by saving and restoring ZFS snapshots and file systems. * Saving ZFS data with archive utilities such as tar and cpio or third-party backup products. http://dlc.sun.com/osol/docs/content/ZFSADMIN/gbchx.html You'll notice that 3/4 mention snapshots. I think them and zfs send/recv are the starting points for getting consistent images of disks. There's no equivalent to dump(8)/restore(8), and so tar and cpio are the main utilities if you want offline stuff: http://www.freebsd.org/doc/handbook/backup-basics.html Until very recently the output format of zfs send was not stabilized, so there was no guarantee that it was readable from one version to the next. I believe that has been fixed in [Open]Solaris, but I haven't been tracking pjd's commits that closely to know about FreeBSD. Hopefully zfs diff will make it into FreeBSD soon-ish, so that it's easier to do incremental backups to previous snapshots/check points. Traversing large file systems is getting really old in this day-and-age, and that one little thing can certainly remove a lot of I/O seeks if you only want to grab the files that have changed recently. See also the best practice guide, which should be generic enough to cover most operating systems: http://tinyurl.com/2gehn4#Recommendations_for_Saving_ZFS_Data http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Recommendations_for_Saving_ZFS_Data -- David Magda Toronto, Canada ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 2010-Nov-23 23:45:43 +1100, Andrew Reilly arei...@bigpond.net.au wrote: zfs send -vR tank/h...@0 | zfs receive -d /backup/snapshots in order to experiment with this strategy. One would then become alarmed when one discovered that the receive mechanism also invoked the mountpoint= parameter of the source filesystem, and the zfs propensity for just doing stuff, and boom: you have a read-only version of your home directory mounted *on top of* your actual home directory... Been there, done that. The undocumented '-u' option to receive will prevent the receive side performing mounts. The poorly documented '-R' option on import allows you to specify an alternative root mountpoint. Once you have done the initial transfer, you can also set 'canmount=noauto' on each fileset (it isn't inherited) to prevent ZFS automounting them on import. BTW, the entire export is performed at the current compression level - recompressing existing data. -- Peter Jeremy pgpJ45k6nVaUR.pgp Description: PGP signature
Re: ZFS backups: retrieving a few files?
On Tue, Nov 23, 2010 at 01:56:01PM +0100, Thomas Ronner wrote: On 11/23/10 1:45 PM, Andrew Reilly wrote: No, I don't like tar, rsync and friends for backups: they don't deal well with hard links, special files or sparse files. rsync -avHxS --delete --numeric-ids /src/. /dst/. Handles sparse files (S) and hard links (H). Never had any trouble with special files. What sort of special files are not handled correctly by rsync? I'd like to know because I'm relying on rsync for backups for years on my home network. I remember having problems with hard links, but it's possible that I wasn't using rsync correctly. Of special files, I don't remember non-dump backups doing well with the unix-domain sockets that were liberally used and tricky to set up right for djb's daemon-tools. Most uses of sockets put them in /tmp, don't get backed up, and they don't need to persist. Dan puts them in /var, and they're expected to persist across reboots, which means they need to be backed-up. Maybe rsync and modern tar handle these OK, but I remember at least one of them getting wedged just trying to read one as a file. The special files in /dev used to be a problem too, but that's gone away, now that we have devfs. I really like dump/restore. I expect that I will miss them. Cheers, -- Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
on 22/11/2010 13:35 Andrew Reilly said the following: The troubling aspect of this plan is that it would seem that grabbing just a few files would require space in the working zpool equivalent to the whole backed-up file system, for the zfs receive of the snapshot. Contrast this with restore, which had a nice interactive (or command line) mode of operation that could retrieve nominated files or directories, requiring only space for the results. Is there any similar tool that can operate on a ZFS send serialization? It would seem that the information must be in there somewhere, but I've not heard of it. Clues? How does everyone else manage this? What I do is keep the snapshots for long enough and just retrieve the files from there. That is, snapshots - for the clumsiness case, backups - for the disaster case. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 11/22/2010 6:35 AM, Andrew Reilly wrote: Dump/restore doesn't work for ZFS. I *think* that I'm running backups in the appropriate equivalent fashion: I take file system snapshots (both absolute == level 0) and relative (incremental), and zfs send those to files on the backup disk. This is actively discouraged, there is no recovery ability when receiving zfs streams so 1 bad bit would invalidate your entire backup. The currently accepted practice is to create a ZFS file system on the backup drive and just keep sending incremental snapshots to it. As long as the backup drive and host system have a snapshot in common you can do incremental transfers. This way you only have to keep the most recent snapshot on the main system and can keep as many as you have space for on the backup drive. You also have direct access to any backed up version of every file. HTH, Jonathan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On Mon, Nov 22, 2010 at 10:26:30AM -0500, Jonathan Stewart wrote: On 11/22/2010 6:35 AM, Andrew Reilly wrote: Dump/restore doesn't work for ZFS. I *think* that I'm running backups in the appropriate equivalent fashion: I take file system snapshots (both absolute == level 0) and relative (incremental), and zfs send those to files on the backup disk. This is actively discouraged, there is no recovery ability when receiving zfs streams so 1 bad bit would invalidate your entire backup. Hmm. Isn't that a problem that also affects the sending snapshots scheme that you describe, below? The currently accepted practice is to create a ZFS file system on the backup drive and just keep sending incremental snapshots to it. As long as the backup drive and host system have a snapshot in common you can do incremental transfers. This way you only have to keep the most recent snapshot on the main system and can keep as many as you have space for on the backup drive. You also have direct access to any backed up version of every file. That sounds like a very cool notion. Not unlike the time-machine scheme. Interesting how different capabilities require going back and re-thinking the problem, rather than just trying to implement the old solution with the new tools. I'll see how I go with it... Cheers, -- Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On 11/22/2010 5:13 PM, Andrew Reilly wrote: On Mon, Nov 22, 2010 at 10:26:30AM -0500, Jonathan Stewart wrote: On 11/22/2010 6:35 AM, Andrew Reilly wrote: Dump/restore doesn't work for ZFS. I *think* that I'm running backups in the appropriate equivalent fashion: I take file system snapshots (both absolute == level 0) and relative (incremental), and zfs send those to files on the backup disk. This is actively discouraged, there is no recovery ability when receiving zfs streams so 1 bad bit would invalidate your entire backup. Hmm. Isn't that a problem that also affects the sending snapshots scheme that you describe, below? When sending the snapshots to a new file system you know right away whether they are bad or not. Otherwise you find out when you are trying to restore and your backups are bad... A send could fail but you can always retry that, you can't retry anything when your trying to restore. Jonathan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ZFS backups: retrieving a few files?
On Nov 22, 2010, at 17:13, Andrew Reilly wrote: The currently accepted practice is to create a ZFS file system on the backup drive and just keep sending incremental snapshots to it. As long as the backup drive and host system have a snapshot in common you can do incremental transfers. This way you only have to keep the most recent snapshot on the main system and can keep as many as you have space for on the backup drive. You also have direct access to any backed up version of every file. That sounds like a very cool notion. Not unlike the time-machine scheme. Interesting how different capabilities require going back and re-thinking the problem, rather than just trying to implement the old solution with the new tools. As noted, saving the output of zfs send isn't very useful and generally not recommended as a backup mechanism. It's come up quite often on Sun/Oracle's zfs-discuss list: http://www.google.com/search?q=zfs+send/receive+as+backup+tool In addition to regular snapshots, also make sure to have an offline backup of some kind (tar, Networker, NetBackup, Amanada, etc.). Errors can propagate to online copies / backups, and if an intruder can penetrate your primary system, there's a good chance they can get to the secondary copy of your data; penetrating a tape on a shelf over the network would be much more challenging. :) Newer versions of ZFS also support a zfs diff command, but I'm not sure if that functionality has made it into FreeBSD yet: http://www.google.com/search?q=zfs+diff Combine diff with some snapshots and scripting, and you can speed up things like tar and rsync a lot since you don't have to traverse the entire file system to find changes. And at the end of the day remember it's not about about backups, but about restores. If you can't restore coherent data in a timely manner it's pointless. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
