Re: Restore UFS snapshot
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: > On Mon, May 28, 2007 at 05:15:19PM +1000, Ian Smith wrote: >> I share Roland's concern about the reliability of any new code designed >> to accomplish the 'full rollback' desired, but of course anything would >> have to undergo incredibly rigorous testing before it would be allowed >> anywhere near even -CURRENT I'm sure, same as softupdates/snapshots has. >> >> That said, I think it is worthwhile finding out whether this is even >> possible with the amount of data and metadata saved in snapshots to >> accomplish their existing utility (which I still find pretty awesome, >> and despite re-reading that section again several times, mysterious ..) > > It should be possible. All the data that is needed is there. But I do > think it will be definitely non-trivial. > > For one thing, you'll have to block all other changes from userland to > the target filesystem while restoring the snapshot. And you'll have to > update other snapshots as well. Seems like a big can of worms to me. I was thinking this should be done while the fs is umounted. As I suggested: 1. umount 2. snap_rollback and 3. mount. Svein Halvor -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) Comment: mailto:[EMAIL PROTECTED] to get my PGP-key iD8DBQFGWr5ahQg3vZGYu0ARAh/iAJ0RdOGSuzgALvRNin2Nd2Xtw3hrmgCguxN4 oKBvLazzHSsN62uas1szqRg= =kOxO -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On Mon, May 28, 2007 at 05:15:19PM +1000, Ian Smith wrote: > > I share Roland's concern about the reliability of any new code designed > to accomplish the 'full rollback' desired, but of course anything would > have to undergo incredibly rigorous testing before it would be allowed > anywhere near even -CURRENT I'm sure, same as softupdates/snapshots has. > > That said, I think it is worthwhile finding out whether this is even > possible with the amount of data and metadata saved in snapshots to > accomplish their existing utility (which I still find pretty awesome, > and despite re-reading that section again several times, mysterious ..) It should be possible. All the data that is needed is there. But I do think it will be definitely non-trivial. For one thing, you'll have to block all other changes from userland to the target filesystem while restoring the snapshot. And you'll have to update other snapshots as well. Seems like a big can of worms to me. > I guess contacting Marshall McKusick might be the quickest way to find > out whether there's sufficient metadata available to do that; he'd know > off the top of his head, I'm sure. So should several other people, I > guess, but more likely those frequenting -fs, -hackers, maybe -arch? Excellent idea. > It may even be possible, assuming restoration of a filesystem to its > state in a snapshot is possible at all, to first save all subsequent > changes in its present state to another snapshot, or a snapshot-like > thing, to allow 'rollforward' again, perhaps .. that would be cute. If you take a snapshot just before you restore the older one, you could do that. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpZGTXEFOArm.pgp Description: PGP signature
Re: Restore UFS snapshot
On Sun, 27 May 2007 11:01:53 -0400 Maxim Khitrov <[EMAIL PROTECTED]> wrote: > On 5/27/07, Roland Smith <[EMAIL PROTECTED]> wrote: > > On Sat, May 26, 2007 at 07:08:31PM -0400, Maxim Khitrov wrote: > > > I don't think he is talking about that. From what I understand about > > > the snapshot system (correct me if I'm wrong) is that a snapshot > > > creates it's own file system by remembering, for example, what the > > > superblock was at the time of the snapshot. After that, the live file > > > system continues on its way keeping track of the snapshot, but > > > modifying its own blocks to account for the changes afterwards. > > > > More or less. You should read §8.7 of McKusick's book, or the paper he > > wrote about it; > > http://www.usenix.org/publications/library/proceedings/usenix99/mckusick.html Interesting thread. I've only read the PDF referred to there, namely http://www.usenix.org/publications/library/proceedings/usenix99/full_papers/mckusick/mckusick.pdf several times, and particularly the snapshot section from p.15 of that, and I'm almost beginning to get it, but can't claim more than that .. > > For instance, every time you change a block on a filesystem that has a > > snapshot, the filesystem checks wether that block is in the snapshot > > already. If not it writes the old contents of that block to the snapshot > > before writing the new contents to the filesystem block. > > > > > I think what Svein wants to do is essentially overwrite a few blocks > > > on the live file system, loosing all references to the changes that > > > have been made and in effect returning the file system to the state it > > > was in when the snapshot was taken. > > > > It's not just a question of references. You'll have to copy all the > > changed blocks back, and change the inodes, et cetera. > > > > > This is different from simply > > > copying the contents of the snapshot back to the disk via an md > > > device. > > > > If you are using something like rsync it's not very much different. > > > > The point is that you're using established and debugged methods and code > > to change the filesystem, instead of writing new tools to do the same. I share Roland's concern about the reliability of any new code designed to accomplish the 'full rollback' desired, but of course anything would have to undergo incredibly rigorous testing before it would be allowed anywhere near even -CURRENT I'm sure, same as softupdates/snapshots has. That said, I think it is worthwhile finding out whether this is even possible with the amount of data and metadata saved in snapshots to accomplish their existing utility (which I still find pretty awesome, and despite re-reading that section again several times, mysterious ..) > > > This way he would restore the snapshot and lose it at the same > > > time, but the operation should be O(1) in theory (time and space), as > > > opposed to O(n) which any normal back-up/restore is. > > > > You don't lose the snapshot untill you delete it. And even then there > > are situations where not all the used space can be reclaimed. If a > > subsequent snapshot was taken, for instance. > > > > The process of undoing the snapshot can't be O(1). Because the time > > needed to create the shapshot isn't either. > > Wait a sec, when you mount a snapshot as a memory disk, does that > memory disk contain the snapshot as well? I haven't done this in a > while, but since the snapshot didn't exist when it was being created, > I don't see how it would. Now when you mount that snapshot, the > superblock is read and from there we find out all the necessary > information about the file-system. This is an O(1) operation is it > not? > > I'm not seeing why this process can't be slightly modified to > roll-back the live file system. There shouldn't be a need to copy any > changed blocks or modify inodes because mounting the snapshot doesn't > require this. The data is already there, all the references to it are > there, these references are just not in the right place for the live > file system to use them. The snapshot superblock contains the same > information regarding the size of the file-system and all other > parameters, so if we were to overwrite the superblock why wouldn't > everything return to the same state you see it in when you mount that > same snapshot? I guess contacting Marshall McKusick might be the quickest way to find out whether there's sufficient metadata available to do that; he'd know off the top of his head, I'm sure. So should several other people, I guess, but more likely those frequenting -fs, -hackers, maybe -arch? > This is why I said that you lose the snapshot itself, because it's not > there when it was taken, and you lose the references to all the > changed blocks of the live file system. Through the data would still > be there on the disk, as far as the snapshot is concerned it's now > free space
Re: Restore UFS snapshot
On Sun, May 27, 2007 at 11:01:53AM -0400, Maxim Khitrov wrote: > > The process of undoing the snapshot can't be O(1). Because the time > > needed to create the shapshot isn't either. > > Wait a sec, when you mount a snapshot as a memory disk, does that > memory disk contain the snapshot as well? Depends of your definition of "contain". Technically, the snaphot is the backing store for the memory disk. > I haven't done this in a > while, but since the snapshot didn't exist when it was being created, > I don't see how it would. I don't know what you are saying here. You have to create a snapshot before you can mount it. > Now when you mount that snapshot, the > superblock is read and from there we find out all the necessary > information about the file-system. This is an O(1) operation is it > not? Mounting the snapshot might be O(1), I don't know. > I'm not seeing why this process can't be slightly modified to > roll-back the live file system. There shouldn't be a need to copy any > changed blocks or modify inodes because mounting the snapshot doesn't > require this. Mounting the snapshot and rolling back the changes in the snapshot are completely different operations. > The data is already there, all the references to it are > there, these references are just not in the right place for the live > file system to use them. The snapshot superblock contains the same > information regarding the size of the file-system and all other > parameters, so if we were to overwrite the superblock why wouldn't > everything return to the same state you see it in when you mount that > same snapshot? A snapshot is basically a copy of the filesystem, without the blocks that haven't changed since the snapshot was made. So overwriting the superblock won't cut it. If you don't want to copy the blocks from the snapshot, you'll have to rewire the inodes for all files that have been changed since the snapshot to use the saved blocks from the snapshot. IMHO it would be safer to copy those blocks then to rewire the fs to use the blocks in the snapshot. A bug in this rewiring code could seriously screw up the filesystem. And it could lead to increased fragmentation. > This is why I said that you lose the snapshot itself, because it's not > there when it was taken, and you lose the references to all the > changed blocks of the live file system. Through the data would still > be there on the disk, as far as the snapshot is concerned it's now > free space that can be overwritten. I mean I understand that there > could be some other updates that need to be done, but I don't see a > need to move or modify any inodes around because everything is already > in place to be used, the live file system just needs to be told how > and where everything is, in the same way the memory disk does this. And how do you "tell the live filesystem where everything is" without modifying inodes? Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpkBewL7cxOh.pgp Description: PGP signature
Re: Restore UFS snapshot
On 5/27/07, Roland Smith <[EMAIL PROTECTED]> wrote: On Sat, May 26, 2007 at 07:08:31PM -0400, Maxim Khitrov wrote: > I don't think he is talking about that. From what I understand about > the snapshot system (correct me if I'm wrong) is that a snapshot > creates it's own file system by remembering, for example, what the > superblock was at the time of the snapshot. After that, the live file > system continues on its way keeping track of the snapshot, but > modifying its own blocks to account for the changes afterwards. More or less. You should read §8.7 of McKusick's book, or the paper he wrote about it; http://www.usenix.org/publications/library/proceedings/usenix99/mckusick.html For instance, every time you change a block on a filesystem that has a snapshot, the filesystem checks wether that block is in the snapshot already. If not it writes the old contents of that block to the snapshot before writing the new contents to the filesystem block. > I think what Svein wants to do is essentially overwrite a few blocks > on the live file system, loosing all references to the changes that > have been made and in effect returning the file system to the state it > was in when the snapshot was taken. It's not just a question of references. You'll have to copy all the changed blocks back, and change the inodes, et cetera. > This is different from simply > copying the contents of the snapshot back to the disk via an md > device. If you are using something like rsync it's not very much different. The point is that you're using established and debugged methods and code to change the filesystem, instead of writing new tools to do the same. > This way he would restore the snapshot and lose it at the same > time, but the operation should be O(1) in theory (time and space), as > opposed to O(n) which any normal back-up/restore is. You don't lose the snapshot untill you delete it. And even then there are situations where not all the used space can be reclaimed. If a subsequent snapshot was taken, for instance. The process of undoing the snapshot can't be O(1). Because the time needed to create the shapshot isn't either. Wait a sec, when you mount a snapshot as a memory disk, does that memory disk contain the snapshot as well? I haven't done this in a while, but since the snapshot didn't exist when it was being created, I don't see how it would. Now when you mount that snapshot, the superblock is read and from there we find out all the necessary information about the file-system. This is an O(1) operation is it not? I'm not seeing why this process can't be slightly modified to roll-back the live file system. There shouldn't be a need to copy any changed blocks or modify inodes because mounting the snapshot doesn't require this. The data is already there, all the references to it are there, these references are just not in the right place for the live file system to use them. The snapshot superblock contains the same information regarding the size of the file-system and all other parameters, so if we were to overwrite the superblock why wouldn't everything return to the same state you see it in when you mount that same snapshot? This is why I said that you lose the snapshot itself, because it's not there when it was taken, and you lose the references to all the changed blocks of the live file system. Through the data would still be there on the disk, as far as the snapshot is concerned it's now free space that can be overwritten. I mean I understand that there could be some other updates that need to be done, but I don't see a need to move or modify any inodes around because everything is already in place to be used, the live file system just needs to be told how and where everything is, in the same way the memory disk does this. The only real problem I see is about how "filesystem checks whether that block is in the snapshot already," as you put it. Is this a flag stored in the inode, or does it involve reading through the snapshot's superblock to see if some inode is referenced by it? In the former case, I suppose the operation wouldn't be O(1) because you'd need to clear all the flags, but you still wouldn't need to move anything around. In the latter case it's even easier. What am I missing from this picture that prevents something like this being done? - Max ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On Sat, May 26, 2007 at 07:08:31PM -0400, Maxim Khitrov wrote: > I don't think he is talking about that. From what I understand about > the snapshot system (correct me if I'm wrong) is that a snapshot > creates it's own file system by remembering, for example, what the > superblock was at the time of the snapshot. After that, the live file > system continues on its way keeping track of the snapshot, but > modifying its own blocks to account for the changes afterwards. More or less. You should read §8.7 of McKusick's book, or the paper he wrote about it; http://www.usenix.org/publications/library/proceedings/usenix99/mckusick.html For instance, every time you change a block on a filesystem that has a snapshot, the filesystem checks wether that block is in the snapshot already. If not it writes the old contents of that block to the snapshot before writing the new contents to the filesystem block. > I think what Svein wants to do is essentially overwrite a few blocks > on the live file system, loosing all references to the changes that > have been made and in effect returning the file system to the state it > was in when the snapshot was taken. It's not just a question of references. You'll have to copy all the changed blocks back, and change the inodes, et cetera. > This is different from simply > copying the contents of the snapshot back to the disk via an md > device. If you are using something like rsync it's not very much different. The point is that you're using established and debugged methods and code to change the filesystem, instead of writing new tools to do the same. > This way he would restore the snapshot and lose it at the same > time, but the operation should be O(1) in theory (time and space), as > opposed to O(n) which any normal back-up/restore is. You don't lose the snapshot untill you delete it. And even then there are situations where not all the used space can be reclaimed. If a subsequent snapshot was taken, for instance. The process of undoing the snapshot can't be O(1). Because the time needed to create the shapshot isn't either. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpRElEnVeolo.pgp Description: PGP signature
Re: Restore UFS snapshot
# umount # snap_rollback *wait 10 seconds* # mount .. and I'm set. I believe it should be possible. And if nothing like that exists, it should be made. I could look into it, but I would have to learn a lot more about the inner workings of the file system first. related: afaik, zfs rollback is working and is planned for freebsd 7.0. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On 5/26/07, Roland Smith <[EMAIL PROTECTED]> wrote: On Sat, May 26, 2007 at 11:59:13PM +0200, Svein Halvor Halvorsen wrote: > Roland Smith wrote: > > You could use rsync instead of tar. That would save time. > > I'm not talking about saving time. But saving CPU time and HDD > stress. However, the disk space issue is a bigger one: rsync would do much less writing than tar. So it would save on "HDD stress", whatever that is. > >> (b) Undo all the bit flipping I have done, since I made the snapshot. > > > > This is what the procedure above does if you replace the tar commands > > with rsync. > > No, because the snapshot will still be in use, and hence all its > bits will be kept intact and read-only. When I use rsync/tar/cpio or > whatever to "undo" changes to a file system, I will in reality copy > these bits to different places on the disk. And until I release the > snapshot (which I very well could, since it would defunct after the > restore process), I will use twice the amount of disk space. You can't restore a previous situation _unless you saved it in some form_. So if you want a possibility to restore stuff, you'll have to keep a copy of it somewhere. Maybe in compressed form, and maybe you can clump changes together in a smart way, but you have to save the bits that you change. Every revision control system (which is effectively what you ask for) uses storage space to keep previous versions of data, although the precise method used for this varies. I don't think he is talking about that. From what I understand about the snapshot system (correct me if I'm wrong) is that a snapshot creates it's own file system by remembering, for example, what the superblock was at the time of the snapshot. After that, the live file system continues on its way keeping track of the snapshot, but modifying its own blocks to account for the changes afterwards. I think what Svein wants to do is essentially overwrite a few blocks on the live file system, loosing all references to the changes that have been made and in effect returning the file system to the state it was in when the snapshot was taken. This is different from simply copying the contents of the snapshot back to the disk via an md device. This way he would restore the snapshot and lose it at the same time, but the operation should be O(1) in theory (time and space), as opposed to O(n) which any normal back-up/restore is. Personally, I think this an entirely reasonable thing to do, and I myself would like to see this kind of functionality. Right now, however, I don't think that it is possible. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: > You can't restore a previous situation _unless you saved it in some > form_. So if you want a possibility to restore stuff, you'll have to > keep a copy of it somewhere. Maybe in compressed form, and maybe you can > clump changes together in a smart way, but you have to save the bits > that you change. Of course! If I'm not clear, you could do better in asking me to clarify, than to assume that I am an idiot. Of course you need to save the data in some form, in order to restore it. I'm not asking for magic. > Every revision control system (which is effectively what you ask for) uses > storage space to keep previous versions of data, although the precise > method used for this varies. Yes, but your suggested solution stores some data twice (at least for some time). And also it involves a lot of reading and writing (even though you could minimize it using rsync). > Disk space is cheap, and getting cheaper. Going through a lot of trouble > to save a few bytes is almost certainly not cost effective. > > And keep in mind that you should really only use the tools that are > available in /rescue. Using a fancy port won't help you if whatever you > did borked /usr/local/bin. :) I am not talking about a backup solution here. I just want an easy way of saving the state, doing something potentially stupid, and then throw away the (stupid) changes real quick and painless. Of course if the disk breaks or something, I will need something else. It doesn't matter too much though; I just wanted to know if something existed or not. Sees it doesn't. Thanks for your time. Maybe I will make something. Maybe not. For now, I will continue to mount the snapshot (as you suggested) Best regards, Svein Halvor -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) Comment: mailto:[EMAIL PROTECTED] to get my PGP-key iD8DBQFGWLyMhQg3vZGYu0ARAq8jAJoDdSRbev54oFKlffjEfAlcv12BfQCgx49L 3Xox5h4HAvgEB+rL1+OLVE8= =AYlv -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On Sat, May 26, 2007 at 11:59:13PM +0200, Svein Halvor Halvorsen wrote: > Roland Smith wrote: > > You could use rsync instead of tar. That would save time. > > I'm not talking about saving time. But saving CPU time and HDD > stress. However, the disk space issue is a bigger one: rsync would do much less writing than tar. So it would save on "HDD stress", whatever that is. > >> (b) Undo all the bit flipping I have done, since I made the snapshot. > > > > This is what the procedure above does if you replace the tar commands > > with rsync. > > No, because the snapshot will still be in use, and hence all its > bits will be kept intact and read-only. When I use rsync/tar/cpio or > whatever to "undo" changes to a file system, I will in reality copy > these bits to different places on the disk. And until I release the > snapshot (which I very well could, since it would defunct after the > restore process), I will use twice the amount of disk space. You can't restore a previous situation _unless you saved it in some form_. So if you want a possibility to restore stuff, you'll have to keep a copy of it somewhere. Maybe in compressed form, and maybe you can clump changes together in a smart way, but you have to save the bits that you change. Every revision control system (which is effectively what you ask for) uses storage space to keep previous versions of data, although the precise method used for this varies. Disk space is cheap, and getting cheaper. Going through a lot of trouble to save a few bytes is almost certainly not cost effective. And keep in mind that you should really only use the tools that are available in /rescue. Using a fancy port won't help you if whatever you did borked /usr/local/bin. :) Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpK7aDwsAnTW.pgp Description: PGP signature
Re: Restore UFS snapshot
On 5/26/07, Svein Halvor Halvorsen <[EMAIL PROTECTED]> wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 pete wright wrote: > hmm...i'm still a little confused as to where you are going. there > are three main way's i've used snapshot's in large (~1PB) > environments, two of which are applicable to you i believe: *snip dump/restore plug* Yes, I understand how I could use dump/restore. But forget about all this. Forget about my reasons for wanting it. All I want to know is whether or not there exists a tool that will let me rollback a snapshot without mounting it, dumping it, or anything like that. Just by flipping some bits in the superblock, or some other small changes to an (unmounted) file system. Something really easy. No extra disk, no excessive copying, no nothing. Just a simple # umount # snap_rollback *wait 10 seconds* # mount .. and I'm set. I believe it should be possible. And if nothing like that exists, it should be made. I could look into it, but I would have to learn a lot more about the inner workings of the file system first. not that i know of, and IMHO for good reason. i would not trust anything of that nature with data that i deemed important enough to snap shot in the first place. -pete -- ~~o0OO0o~~ Pete Wright www.nycbug.org NYC's *BSD User Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 pete wright wrote: > hmm...i'm still a little confused as to where you are going. there > are three main way's i've used snapshot's in large (~1PB) > environments, two of which are applicable to you i believe: *snip dump/restore plug* Yes, I understand how I could use dump/restore. But forget about all this. Forget about my reasons for wanting it. All I want to know is whether or not there exists a tool that will let me rollback a snapshot without mounting it, dumping it, or anything like that. Just by flipping some bits in the superblock, or some other small changes to an (unmounted) file system. Something really easy. No extra disk, no excessive copying, no nothing. Just a simple # umount # snap_rollback *wait 10 seconds* # mount .. and I'm set. I believe it should be possible. And if nothing like that exists, it should be made. I could look into it, but I would have to learn a lot more about the inner workings of the file system first. Svein Halvor -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) Comment: mailto:[EMAIL PROTECTED] to get my PGP-key iD8DBQFGWK/ChQg3vZGYu0ARAoLIAJoDCeyZf/lsO/sj0HbZtosKs4i/lgCghohK Uc+zpgqsxUNVCV5yd/x0BQ0= =J6Az -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: > You could use rsync instead of tar. That would save time. I'm not talking about saving time. But saving CPU time and HDD stress. However, the disk space issue is a bigger one: >> (b) Undo all the bit flipping I have done, since I made the snapshot. > > This is what the procedure above does if you replace the tar commands > with rsync. No, because the snapshot will still be in use, and hence all its bits will be kept intact and read-only. When I use rsync/tar/cpio or whatever to "undo" changes to a file system, I will in reality copy these bits to different places on the disk. And until I release the snapshot (which I very well could, since it would defunct after the restore process), I will use twice the amount of disk space. Svein Halvor -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) Comment: mailto:[EMAIL PROTECTED] to get my PGP-key iD8DBQFGWK2vhQg3vZGYu0ARAk/5AJ9QksQAbmwKTJLkwKGhISMpMvOEZgCgwG5u s7bYTdMu9DEIylAhTCeepzI= =5cD3 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On 5/26/07, Svein Halvor Halvorsen <[EMAIL PROTECTED]> wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: >>> You can mount the snapshot, and then copy the files back to the original fs. >>> Note that cp can preserve flags, but not ACLs AFAIK. >> Yes, I know that this is possible. However, it's a lot of work. > > Huh? > > Suppose you did 'mksnap_ffs /usr /usr/.snap/20070526' > > Then all you have to is something like: > > # mdconfig -a -t vnode -f /usr/.snap/20070526 -u 0 > # mount /dev/md0 /mnt/snapshot > # cd /usr > # tar cf - /mnt/snapshot/* |tar xpf - > # umount /mnt/snapshot > # mdconfig -d -u 0 > > How much easier could it be? You could easily create a script for this > as well. Let me clarify: It is a lot of work for the computer, for the hdd. >> There should be some straightforward way of rolling back to a >> snapshot, since the files and all the file system structure are >> already there. Also, there might not be room on the disk for it. > > Snapshots take up room as well. But the snapshot is already made. Again, let me clarify: At some point in time, my file system is filled with random* bits. I then make a snapshot. - From now on, all bits** that I flip will be take up an extra bit of space. Then, after changing lots of bits, I decide I wanted the old data back, as the file system was before I started to flip bits. Now, I could either: (a) Flip alot more bits, by making copies of the snapshotted bits over some free area of the disk, or (b) Undo all the bit flipping I have done, since I made the snapshot. In (a) I will have two copies of all the bits that has changed since the original snapshot, while in (b) I am back to where i were before the snapshot. Does this make any sense? Have I not understood this correctly? hmm...i'm still a little confused as to where you are going. there are three main way's i've used snapshot's in large (~1PB) environments, two of which are applicable to you i believe: 1) dump(8) file system after snapshot, not only for backup/DR purposes - but to insure that you have a valid disk image of your critical filesystem before doing something risky (installworld etc.). in this case dump to a scratch volume 2) restore(8) dumped filesystem image if something bad happens, otherwise let tmpwatch clean remove the dump at a later date. while this may require more space, it does give you a reasonable amount of certainty that the disk image is valid and consistent (esp. pertinent for frequently modified data sets - let's say LDAP databases). now, here is an easy way to go - that should work for static dataset's: an installworld goes bad and /usr/bin is borked: $ tar cvpf - /usr/bin/.snap/ | (cd /usr/bin; tar xvpf -) or something similar. you could use rsync, but that would give you uneeded overhead IMHO. -p -- ~~o0OO0o~~ Pete Wright www.nycbug.org NYC's *BSD User Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On Sat, May 26, 2007 at 10:30:13PM +0200, Svein Halvor Halvorsen wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Roland Smith wrote: > >>> You can mount the snapshot, and then copy the files back to the original > >>> fs. > >>> Note that cp can preserve flags, but not ACLs AFAIK. > >> Yes, I know that this is possible. However, it's a lot of work. > > > > Huh? > > > > Suppose you did 'mksnap_ffs /usr /usr/.snap/20070526' > > > > Then all you have to is something like: > > > > # mdconfig -a -t vnode -f /usr/.snap/20070526 -u 0 > > # mount /dev/md0 /mnt/snapshot > > # cd /usr > > # tar cf - /mnt/snapshot/* |tar xpf - > > # umount /mnt/snapshot > > # mdconfig -d -u 0 > > > > How much easier could it be? You could easily create a script for this > > as well. > > Let me clarify: It is a lot of work for the computer, for the hdd. You could use rsync instead of tar. That would save time. > (b) Undo all the bit flipping I have done, since I made the snapshot. This is what the procedure above does if you replace the tar commands with rsync. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpeK5vgqe8Oe.pgp Description: PGP signature
Re: Restore UFS snapshot
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: >>> You can mount the snapshot, and then copy the files back to the original fs. >>> Note that cp can preserve flags, but not ACLs AFAIK. >> Yes, I know that this is possible. However, it's a lot of work. > > Huh? > > Suppose you did 'mksnap_ffs /usr /usr/.snap/20070526' > > Then all you have to is something like: > > # mdconfig -a -t vnode -f /usr/.snap/20070526 -u 0 > # mount /dev/md0 /mnt/snapshot > # cd /usr > # tar cf - /mnt/snapshot/* |tar xpf - > # umount /mnt/snapshot > # mdconfig -d -u 0 > > How much easier could it be? You could easily create a script for this > as well. Let me clarify: It is a lot of work for the computer, for the hdd. >> There should be some straightforward way of rolling back to a >> snapshot, since the files and all the file system structure are >> already there. Also, there might not be room on the disk for it. > > Snapshots take up room as well. But the snapshot is already made. Again, let me clarify: At some point in time, my file system is filled with random* bits. I then make a snapshot. - From now on, all bits** that I flip will be take up an extra bit of space. Then, after changing lots of bits, I decide I wanted the old data back, as the file system was before I started to flip bits. Now, I could either: (a) Flip alot more bits, by making copies of the snapshotted bits over some free area of the disk, or (b) Undo all the bit flipping I have done, since I made the snapshot. In (a) I will have two copies of all the bits that has changed since the original snapshot, while in (b) I am back to where i were before the snapshot. Does this make any sense? Have I not understood this correctly? Best regards, Svein Halvor *) well, not random, but they might just as well be for the sake of the argument **) actually not bits either, but blocks or whatever. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) Comment: mailto:[EMAIL PROTECTED] to get my PGP-key iD8DBQFGWJjUhQg3vZGYu0ARAofgAJ9QS1pPyYEmeQ8TkgYR7HbptZ014QCgqmkR 1dr8wcQV0qhR9KH7VlG/4Q0= =ssrQ -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On 5/26/07, Svein Halvor Halvorsen <[EMAIL PROTECTED]> wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: >> Is it possible to rollback a file system snapshot, i.e. restore the >> file system to the state it was in at the time a mksnap_ffs command >> was issued? > > You can mount the snapshot, and then copy the files back to the original fs. > Note that cp can preserve flags, but not ACLs AFAIK. Yes, I know that this is possible. However, it's a lot of work. There should be some straightforward way of rolling back to a snapshot, since the files and all the file system structure are already there. Also, there might not be room on the disk for it. well, if you are using snapshot's you already have most likely calculated the overhead that the snapshot(s) will take - so i'm a little confused at to the lack of room available for the snapshot. it's not uncommon to have hourly, daily, weekly snapshot's of given volumes. >> User scenario: >> >> Before a major upgrade (eg. releng->current, portupgrade -a, etc), >> it would be nice to mksnap_ffs, and then after the upgrade be able >> to either delete the snapshot if all went well, or rollback to the >> snapshot. > > You should use dump(8) in this case. Create level 0 dumps of your > filesystems and store them somewhere. You can dump live filesystems with > dump's -L flag. > > If you botch the upgrade, you can use restore(8) to revert your > filesystems to the situation before the upgrade. > > Note that you should really make regular dumps of your filesystems as > backups anyway! This is also beyond the point, although I appreciate that you suggest alternative ways to meet my objectives. dump/restore would also require additional disk space. I do actually backup my data on a regular basis, but not all of my computers really need external backup, as I could stand some downtime. However, if I could easily make a snapshot, and then either roll back or delete it afterwards, it would be a nice compromise between security and effort. And also: it seems it should be possible to do this. If not, I might want to make a tool for it. they handbook has a pretty decent example of how to use dump along side mksnap_ffs - and it seems pretty robust to me. when dealing with whole filesystems and important data i think dump(8) is really the way to go as much work has been put into ensuring that you end up with a consistent image on disk. having said that - i see no reason why one couldn't write a wrapper around dump(8) and mksnap_ffs. -p -- ~~o0OO0o~~ Pete Wright www.nycbug.org NYC's *BSD User Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On Sat, May 26, 2007 at 09:05:07PM +0200, Svein Halvor Halvorsen wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Roland Smith wrote: > >> Is it possible to rollback a file system snapshot, i.e. restore the > >> file system to the state it was in at the time a mksnap_ffs command > >> was issued? > > > > You can mount the snapshot, and then copy the files back to the original fs. > > Note that cp can preserve flags, but not ACLs AFAIK. > > Yes, I know that this is possible. However, it's a lot of work. Huh? Suppose you did 'mksnap_ffs /usr /usr/.snap/20070526' Then all you have to is something like: # mdconfig -a -t vnode -f /usr/.snap/20070526 -u 0 # mount /dev/md0 /mnt/snapshot # cd /usr # tar cf - /mnt/snapshot/* |tar xpf - # umount /mnt/snapshot # mdconfig -d -u 0 How much easier could it be? You could easily create a script for this as well. > There should be some straightforward way of rolling back to a > snapshot, since the files and all the file system structure are > already there. Also, there might not be room on the disk for it. Snapshots take up room as well. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgp8ndcyeNNPt.pgp Description: PGP signature
Re: Restore UFS snapshot
Svein Halvor Halvorsen <[EMAIL PROTECTED]> writes: > This is also beyond the point, although I appreciate that you > suggest alternative ways to meet my objectives. dump/restore would > also require additional disk space. Not as elegant as your idea, but you can always dump from the snapshot and restore back on the filesystem. You can't use a "pristine" restore(8), and you need the extra space on the same filesystem, but it will work. Your basic idea seems quite workable as far as I can see, so it would be a Simple Matter of Programming to get it to work. [Note that since the copy-on-write mapping only goes one way, the code to do this will have to walk the whole snapshot filesystem, rebuilding inode status (and cleaning up ones that didn't exist in the snapshot) as it goes. A fair amount of work, even if nothing is tricky in theory.] Be well. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roland Smith wrote: >> Is it possible to rollback a file system snapshot, i.e. restore the >> file system to the state it was in at the time a mksnap_ffs command >> was issued? > > You can mount the snapshot, and then copy the files back to the original fs. > Note that cp can preserve flags, but not ACLs AFAIK. Yes, I know that this is possible. However, it's a lot of work. There should be some straightforward way of rolling back to a snapshot, since the files and all the file system structure are already there. Also, there might not be room on the disk for it. >> User scenario: >> >> Before a major upgrade (eg. releng->current, portupgrade -a, etc), >> it would be nice to mksnap_ffs, and then after the upgrade be able >> to either delete the snapshot if all went well, or rollback to the >> snapshot. > > You should use dump(8) in this case. Create level 0 dumps of your > filesystems and store them somewhere. You can dump live filesystems with > dump's -L flag. > > If you botch the upgrade, you can use restore(8) to revert your > filesystems to the situation before the upgrade. > > Note that you should really make regular dumps of your filesystems as > backups anyway! This is also beyond the point, although I appreciate that you suggest alternative ways to meet my objectives. dump/restore would also require additional disk space. I do actually backup my data on a regular basis, but not all of my computers really need external backup, as I could stand some downtime. However, if I could easily make a snapshot, and then either roll back or delete it afterwards, it would be a nice compromise between security and effort. And also: it seems it should be possible to do this. If not, I might want to make a tool for it. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) Comment: mailto:[EMAIL PROTECTED] to get my PGP-key iD4DBQFGWITghQg3vZGYu0ARAjIeAJwIe7+pbMw62dHClFo1r6R6byUKaQCYzWx3 QcIl0qBiYsKdyytwxzVHww== =OFQk -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Restore UFS snapshot
On Sat, May 26, 2007 at 06:48:52PM +0200, Svein Halvor Halvorsen wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hello list! > > Is it possible to rollback a file system snapshot, i.e. restore the > file system to the state it was in at the time a mksnap_ffs command > was issued? You can mount the snapshot, and then copy the files back to the original fs. Note that cp can preserve flags, but not ACLs AFAIK. > User scenario: > > Before a major upgrade (eg. releng->current, portupgrade -a, etc), > it would be nice to mksnap_ffs, and then after the upgrade be able > to either delete the snapshot if all went well, or rollback to the > snapshot. You should use dump(8) in this case. Create level 0 dumps of your filesystems and store them somewhere. You can dump live filesystems with dump's -L flag. If you botch the upgrade, you can use restore(8) to revert your filesystems to the situation before the upgrade. Note that you should really make regular dumps of your filesystems as backups anyway! Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpyhVuYhuQko.pgp Description: PGP signature
