Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
I believe the question of Ned and the answers given have more far reaching consequences than has been discussed so far. When I read this thread I thought there was an easy solution to deleting files from a snapshot by using clones instead. Clones are a writable copy so you should be able to delete files selectively. However it turns out that this is not possible: - Clones can only be made from snapshots - Snapshots can only be deleted if there is only one dataset that directly derives from them. The snapshot can not be destroyed as long as both the original file system and the clone live. And the "big file" that Ned wants to delete will still survive in the snapshot even if deleted from the clone. Example: zfs create t1 zfs snapshot t...@s1 zfs snapshot t...@s2 zfs snapshot t...@s3 zfs clone t...@s2 t2 zfs snapshot t...@s4 creates datasets with the following origin graph owned by t1: s1 <- s2 <- s3 <- t1 owned by t2: (s2) <- s4 <- t2 Any snapshot can be destroyed except s2. And t2 can be destroyed but not t1 since destroying a filesystem implicitly destroys all owned snapshots. zfs promote can be used to swap the roles of t1 and t2. Thus it seems that clones are not at all independent copies, because all copies of a filesystem are always linked via an older snapshot. Given the central role of read-only snapshots to the design, my fear would be that sharing files across independent file systems is impossible. Then again there is pool wide deduplication which seems to have no problem sharing blocks across independent filesystems. Creating a new filesystem and copying all files over should result in two filesystems sharing the same blocks. This very same result should also be achievable without having to copy the data. And with independent clones Neds use case would be possible. ZFS has created a world where setting up several filesystems even within a single users home directory is encouraged and has been made very easy. I would find it highly desirable if files could be moved across filesystem boundaries without the need to revert to a copy & remove. This is both due to the difference in performance as well as due to the waste of space if the moved file is already part of a snapshot of the old filesystem. Can somebody with more knowledge about the ZFS internals say something on the possibility of independent clones and sharing / moving between filesystems? Michael Bosch -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
On 04/18/10 01:25 AM, Edward Ned Harvey wrote: From: Ian Collins [mailto:i...@ianshome.com] But is a fundamental of zfs: snapshot A read-only version of a file system or volume at a given point in time. It is specified as filesys...@name or vol...@name. Erik Trimble's assessment that it would be difficult to implement is much more valuable. Because the above quote is trivial to change. It's a lot more than that, to quote Eric, it is "a fundamental design decision made when ZFS was started". Snapshots have always been read-only and have defined that way from day one in all publications describing ZFS. Changing words my be easy, but this more than changing words. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
> From: Ian Collins [mailto:i...@ianshome.com] > > But is a fundamental of zfs: > > snapshot > > A read-only version of a file system or volume at a > given point in time. It is specified as filesys...@name > or vol...@name. Erik Trimble's assessment that it would be difficult to implement is much more valuable. Because the above quote is trivial to change. The source code to implement the change of behavior is the only hard part. I've seen many changes in the man page take place over the last year, to reflect the creation / correction of new features as time moves on. Erik, or somebody else: Are you involved in ZFS development? Can you say authoritatively the complexity of changes required for "zfs destroy somef...@snap" is simply not reasonable to implement? Or perhaps, it could be done but there would need to be sufficient momentum behind it in the form of commercial customer demand, or something like that? (which is not likely to happen.) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
Ian Collins wrote: On 04/17/10 12:56 PM, Edward Ned Harvey wrote: From: Erik Trimble [mailto:erik.trim...@oracle.com] Sent: Friday, April 16, 2010 7:35 PM Doesn't that defeat the purpose of a snapshot? Eric hits the nail right on the head: you *don't* want to support such a "feature", as it breaks the fundamental assumption about what a snapshot is (and represents). Ok, point taken, but what you've stated is just an opinion. It's not a fundamental necessity, or a mathematical necessity, or impossible to think otherwise, that a snapshot is 100% and always will be immutable. But is a fundamental of zfs: snapshot A read-only version of a file system or volume at a given point in time. It is specified as filesys...@name or vol...@name. As Ian said. Also, take a look at the old conversation about writeable snapshots - sure, it's my /opinion/ that snapshots should be read-only, but it's also a fundamental design decision made when ZFS was started. There's actually non-trivial assumptions in the code about the immutability of a snapshot, and I'd not want to go down that path of changing a fundamental assumption of the whole system. The problem with mutable snapshots is that you've moved over into the realm of a "versioning" filesystem, and that's a whole 'nother rats nest of complicated issues. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
On 04/17/10 12:56 PM, Edward Ned Harvey wrote: From: Erik Trimble [mailto:erik.trim...@oracle.com] Sent: Friday, April 16, 2010 7:35 PM Doesn't that defeat the purpose of a snapshot? Eric hits the nail right on the head: you *don't* want to support such a "feature", as it breaks the fundamental assumption about what a snapshot is (and represents). Ok, point taken, but what you've stated is just an opinion. It's not a fundamental necessity, or a mathematical necessity, or impossible to think otherwise, that a snapshot is 100% and always will be immutable. But is a fundamental of zfs: snapshot A read-only version of a file system or volume at a given point in time. It is specified as filesys...@name or vol...@name. IMHO, for some people in some situations the assumption that a snapshot is identical to the way the FS was at some given time can be valuable. However: If your only option is to destroy the whole snapshot, in order to free up disk space occupied by some files in the snapshot ... Destroying the whole snapshot for some people in some situations can be even less desirable than destroying the subset of files you want freed from disk. There could be value in the ability to destroy some subset of files without destroying the whole snapshot. I can see your point, it can be really annoying when there's a very large file you want to delete to free space locked up in snapshots. I've been there and it was a pain. Now I use nested filesystems for storing media files, so removing snapshots is more manageable. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
> From: Nicolas Williams [mailto:nicolas.willi...@oracle.com] > > you should send your snapshots to backup and clean them out from > time to time anyways. When using ZFS as a filesystem in a fileserver, the desired configuration such as auto-snapshots is something like: Every 15 mins for the most recent hour Every hour for the most recent 24 hours Every day for the most recent 30 days Or something like that. And all snapshots older than 30 days are automatically destroyed. A good sysadmin in this situation will send snaps to backup, and also allow the snaps to rotate automatically. There need not be an absolute, "Snapshots exist for the sole purpose of staging for removable media backup." Snapshots, in and of themselves, are useful as an augmentation to regular backups. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
> From: Erik Trimble [mailto:erik.trim...@oracle.com] > Sent: Friday, April 16, 2010 7:35 PM > > > Doesn't that defeat the purpose of a snapshot? > > > Eric hits > the > nail right on the head: you *don't* want to support such a "feature", > as it breaks the fundamental assumption about what a snapshot is (and > represents). Ok, point taken, but what you've stated is just an opinion. It's not a fundamental necessity, or a mathematical necessity, or impossible to think otherwise, that a snapshot is 100% and always will be immutable. IMHO, for some people in some situations the assumption that a snapshot is identical to the way the FS was at some given time can be valuable. However: If your only option is to destroy the whole snapshot, in order to free up disk space occupied by some files in the snapshot ... Destroying the whole snapshot for some people in some situations can be even less desirable than destroying the subset of files you want freed from disk. There could be value in the ability to destroy some subset of files without destroying the whole snapshot. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
On Fri, Apr 16, 2010 at 01:56:07PM -0400, Edward Ned Harvey wrote: > The typical problem scenario is: Some user or users fill up the filesystem. > They rm some files, but disk space is not freed. You need to destroy all > the snapshots that contain the deleted files, before disk space is available > again. > > It would be nice if you could rm files from snapshots, without needing to > destroy the whole snapshot. > > Is there any existing work or solution for this? See the archives. See the other replies to you already. Short version: no. However, a script to find all the snapshots that you'd have to delete in order to delete some file might be useful, but really, only marginally so: you should send your snapshots to backup and clean them out from time to time anyways. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
Eric D. Mudama wrote: On Fri, Apr 16 at 13:56, Edward Ned Harvey wrote: The typical problem scenario is: Some user or users fill up the filesystem. They rm some files, but disk space is not freed. You need to destroy all the snapshots that contain the deleted files, before disk space is available again. It would be nice if you could rm files from snapshots, without needing to destroy the whole snapshot. Is there any existing work or solution for this? Doesn't that defeat the purpose of a snapshot? If this is a real problem, I think that it calls for putting that user's files in a separate filesystem that can have its snapshots managed with a specific policy for addressing the usage model. --eric There was a discussion on this a couple of months ago, and Eric hits the nail right on the head: you *don't* want to support such a "feature", as it breaks the fundamental assumption about what a snapshot is (and represents). AFAIK, snapshots will remain forever read-only, with any action acting on the snapshot as a whole (delete/promote/clone, et al.), and there is no plan to ever change this design. It's unfortunate that people can get themselves into a situation where they might need this ability, but, honestly, it's a self-inflicted wound, and there's only so much designers can do to prevent people from shooting themselves in the foot. It's also not a fatal problem - deleting snapshots is still possible, it's just not the "simplest" method of recovery. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZFS better: rm files/directories from snapshots
On Fri, Apr 16 at 13:56, Edward Ned Harvey wrote: The typical problem scenario is: Some user or users fill up the filesystem. They rm some files, but disk space is not freed. You need to destroy all the snapshots that contain the deleted files, before disk space is available again. It would be nice if you could rm files from snapshots, without needing to destroy the whole snapshot. Is there any existing work or solution for this? Doesn't that defeat the purpose of a snapshot? If this is a real problem, I think that it calls for putting that user's files in a separate filesystem that can have its snapshots managed with a specific policy for addressing the usage model. --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Making ZFS better: rm files/directories from snapshots
The typical problem scenario is: Some user or users fill up the filesystem. They rm some files, but disk space is not freed. You need to destroy all the snapshots that contain the deleted files, before disk space is available again. It would be nice if you could rm files from snapshots, without needing to destroy the whole snapshot. Is there any existing work or solution for this? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
