Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
> They are defined but unused in 2.6.19, right? I can't see anywhere > in the 2.6.19 ext2/3/4/reiser trees that actually those flags, > including setting and retrieving them from disk. JFS i can see > sets, clears and retreives them, but not the fielsystems you > mention. Though I might just be blind. ;) Well, at least Ext2/3/4 and JFS do store these attributes and retrieve them from the disk. However, not a single one of these file systems supports the related functionality (secure deletion or a trash-bin). Clearly, this is confusing for the users. > If all we need to add to XFS is support for those flags, then XFS > support would be trivial to add. > > Oh, damn. I take that back. We're almost out of flag space in the on > disk inode - these two flags would use the last 2 flag bits so this > may require an on disk inode format change in XFS. This will be > a little more complex than I first thought, but not impossible > as we already support two on-disk inode format versions. If you do not see any obvious use for these two bits in the near future and ready to sacrifice them (or at least one of them) for secure deletion and trash-bin attributes we would be more than happy to add the actual functionality to XFS as well. > I'd certainly like to see new functionality like this added at the > VFS rather than in any particular filesystem. Don't know about > anyone else, though > > > I guess, the right thing to do would be to move the common trash-bin > > (tb.c and tb.h) code into the /fs and /include/linux directories. > > And call them trashbin.[ch] ;) In fact, we have already done this :-) > > Also, VFS would require just a couple of trivial changes to support > > something like '-o trashbin' mount-time option for all file systems. > > In addition, individual file systems may support per-file attributes for > > this (e.g., ext2/3/4). > > Sounds like a good idea to me. Thank you! So we will submit the corresponding patches soon. We will start from the trashbin.c and patches for the individual file systems. A little later we will submit a VFS patch to add support for the '-o trashbin' mount option. Nikolai Joukov -- Filesystems and Storage Laboratory Stony Brook University - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Thu, 2006-12-07 at 12:44 +1100, David Chinner wrote: > Maybe we should be using EAs for this sort of thing instead of flags > on the inode? If we keep adding inode flags for generic features > then we are going to force more than just XFS into inode format > changes eventually You do need to be judicious in what you add, but we got up to here (well over 10 years) with 16 bits worth of space - increasing to 32 bits is going to last a fairly long time... I wouldn't switch to a different scheme until that point (which is also the point where the system calls are going to have to change anyway). cheers. -- Nathan - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Wed, Dec 06, 2006 at 07:14:58PM -0800, Nicholas Miell wrote: > On Thu, 2006-12-07 at 13:49 +1100, David Chinner wrote: > > On Wed, Dec 06, 2006 at 09:35:30PM -0500, Josef Sipek wrote: > > > On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote: > > > > Maybe we should be using EAs for this sort of thing instead of flags > > > > on the inode? If we keep adding inode flags for generic features > > > > then we are going to force more than just XFS into inode format > > > > changes eventually > > > > > > Aren't EAs slow? Maybe not on XFS but on other filesystems... > > > > Only when they don't fit in the inode itself and extra > > disk seeks are needed to retrieve them. > > > > Cheers, > > > > Dave. > > Also keep in mind that the EA doesn't actually have to have a physical > representation on disk (or, rather, it does, but it doesn't need to be > the same representation used by EAs in the user namespace). > > This means that if one of those slow EA filesystems still has room for > flags in the inode, it can synthesize the EA on demand. Interesting point. Filesystems such as XFS would be unaffected (at least with attr v1, not sure how attr2 does things in XFS wrt EAs.) Another question is, suppose that filesystem x is one of the slow EA filesystems, exposing this whiteout inode flag as EA would require fs x to be compiled with EA support. As a matter of fact, it would require all the filesystems to have EA support. Sure, overall it is less work for me, but I am more interested in doing the Right Thing(tm) - which may be something completely different from EA or inode flag (e.g., what Ted Tso suggested at OLS - effectively a metadata-only ondisk format for unionfs.) > This is even preferable to ioctls for the interface to new filesystem > metadata -- if a backup or archive program knows how to store EAs, it > will be able to save and restore any new exotic metadata without any > extra effort. Agreed. Josef "Jeff" Sipek. -- UNIX is user-friendly ... it's just selective about who it's friends are - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Wed, 2006-12-06 at 20:11 +1100, David Chinner wrote:
> ...
> If all we need to add to XFS is support for those flags, then XFS
> support would be trivial to add.
>
> Oh, damn. I take that back. We're almost out of flag space in the on
> disk inode - these two flags would use the last 2 flag bits so this
> may require an on disk inode format change in XFS. This will be
> a little more complex than I first thought, ...
It should be OK - you can do it without an inode version revision
if you take a second 16 bits for "di_flags2" from here...
xfs_dinode_core {
...
__uint8_t di_pad[8]; /* unused, zeroed space */
Its guaranteed zeroed initially (i.e. all flags unset) and the XFS
get/set flags APIs are 32 bits, so you should be OK there.
Also, it may also be possible to reclaim di_onlink at some point (maybe
now, since 16 bits would be good here) if mkfs.xfs is changed to always
create v2 inodes (dynamic conversion ATM IIRC)... not 100% sure though,
needs more code analysis.
cheers.
--
Nathan
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Thu, 2006-12-07 at 13:49 +1100, David Chinner wrote: > On Wed, Dec 06, 2006 at 09:35:30PM -0500, Josef Sipek wrote: > > On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote: > > > Maybe we should be using EAs for this sort of thing instead of flags > > > on the inode? If we keep adding inode flags for generic features > > > then we are going to force more than just XFS into inode format > > > changes eventually > > > > Aren't EAs slow? Maybe not on XFS but on other filesystems... > > Only when they don't fit in the inode itself and extra > disk seeks are needed to retrieve them. > > Cheers, > > Dave. Also keep in mind that the EA doesn't actually have to have a physical representation on disk (or, rather, it does, but it doesn't need to be the same representation used by EAs in the user namespace). This means that if one of those slow EA filesystems still has room for flags in the inode, it can synthesize the EA on demand. This is even preferable to ioctls for the interface to new filesystem metadata -- if a backup or archive program knows how to store EAs, it will be able to save and restore any new exotic metadata without any extra effort. -- Nicholas Miell <[EMAIL PROTECTED]> - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Wed, Dec 06, 2006 at 09:35:30PM -0500, Josef Sipek wrote: > On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote: > > Maybe we should be using EAs for this sort of thing instead of flags > > on the inode? If we keep adding inode flags for generic features > > then we are going to force more than just XFS into inode format > > changes eventually > > Aren't EAs slow? Maybe not on XFS but on other filesystems... Only when they don't fit in the inode itself and extra disk seeks are needed to retrieve them. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote: > Maybe we should be using EAs for this sort of thing instead of flags > on the inode? If we keep adding inode flags for generic features > then we are going to force more than just XFS into inode format > changes eventually Aren't EAs slow? Maybe not on XFS but on other filesystems... Josef "Jeff" Sipek. -- Linux, n.: Generous programmers from around the world all join forces to help you shoot yourself in the foot for free. - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Wed, Dec 06, 2006 at 07:56:19PM -0500, Josef Sipek wrote: > On Wed, Dec 06, 2006 at 08:11:00PM +1100, David Chinner wrote: > > They are defined but unused in 2.6.19, right? I can't see anywhere > > in the 2.6.19 ext2/3/4/reiser trees that actually those flags, > > including setting and retrieving them from disk. JFS i can see > > sets, clears and retreives them, but not the fielsystems you > > mention. Though I might just be blind. ;) > > > > If all we need to add to XFS is support for those flags, then XFS > > support would be trivial to add. > > > > Oh, damn. I take that back. We're almost out of flag space in the on > > disk inode - these two flags would use the last 2 flag bits so this > > may require an on disk inode format change in XFS. This will be > > a little more complex than I first thought, but not impossible > > as we already support two on-disk inode format versions. > > Hrm. I was toying around with the idea of using a flag to mark inodes as > whiteouts (similar to what BSD does) for Unionfs. I remember that Jan Blunck > tried similar thing in his implementation of VFS unionfs mounts. > > I am not entirely convinced that whiteout inode flag is the right way to do > things, but I'm just raising this now as I wouldn't want to wait for new > ondisk format for XFS to say that Unionfs supports XFS. (Assuming that it is > the right approach.) ;-) Maybe we should be using EAs for this sort of thing instead of flags on the inode? If we keep adding inode flags for generic features then we are going to force more than just XFS into inode format changes eventually Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Wed, Dec 06, 2006 at 08:11:00PM +1100, David Chinner wrote: > They are defined but unused in 2.6.19, right? I can't see anywhere > in the 2.6.19 ext2/3/4/reiser trees that actually those flags, > including setting and retrieving them from disk. JFS i can see > sets, clears and retreives them, but not the fielsystems you > mention. Though I might just be blind. ;) > > If all we need to add to XFS is support for those flags, then XFS > support would be trivial to add. > > Oh, damn. I take that back. We're almost out of flag space in the on > disk inode - these two flags would use the last 2 flag bits so this > may require an on disk inode format change in XFS. This will be > a little more complex than I first thought, but not impossible > as we already support two on-disk inode format versions. Hrm. I was toying around with the idea of using a flag to mark inodes as whiteouts (similar to what BSD does) for Unionfs. I remember that Jan Blunck tried similar thing in his implementation of VFS unionfs mounts. I am not entirely convinced that whiteout inode flag is the right way to do things, but I'm just raising this now as I wouldn't want to wait for new ondisk format for XFS to say that Unionfs supports XFS. (Assuming that it is the right approach.) ;-) Josef "Jeff" Sipek. -- I already backed up the box once, I can do it again! - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Tue, Dec 05, 2006 at 11:41:28AM -0500, Nikolai Joukov wrote: > > > As we promised on the linux-ext4 list on October 31, here is the patch > > > that adds secure deletion via a trash-bin functionality for ext4. It is a > > > compromise solution that combines secure deletion with the trash-bin > > > support > > > (the latter had been requested by even more people than the former :-). > > > > Given that almost all of the code for this uses vfs interfaces and > > only a couple of simple filesystem hooks, is there any reason for > > this being ext4 specific? i.e. why aren't you hooking the vfs layer > > so we get a single undelete/secure delete implementation for all > > filesystems? > > You are right. Actually, we mentioned it as a benefit number 4 of this > approach in the original post. Most of the code is not > file-system--specific and can be used by any other (all other?) file > system(s). The only complication is that only ext2/3/4 and reiser file > systems already support the per-file secure deletion and undelete > attributes. They are defined but unused in 2.6.19, right? I can't see anywhere in the 2.6.19 ext2/3/4/reiser trees that actually those flags, including setting and retrieving them from disk. JFS i can see sets, clears and retreives them, but not the fielsystems you mention. Though I might just be blind. ;) If all we need to add to XFS is support for those flags, then XFS support would be trivial to add. Oh, damn. I take that back. We're almost out of flag space in the on disk inode - these two flags would use the last 2 flag bits so this may require an on disk inode format change in XFS. This will be a little more complex than I first thought, but not impossible as we already support two on-disk inode format versions. > Since ext4 is in early development now, we believe it'd be easier to get > such code into ext4 than into the main-line VFS. If there's enough > interested among the kernel maintainers, we'd be happy to move this > functionality to the VFS and provide f/s hooks for > secure-deletion/trash-bin. I'd certainly like to see new functionality like this added at the VFS rather than in any particular filesystem. Don't know about anyone else, though > I guess, the right thing to do would be to move the common trash-bin > (tb.c and tb.h) code into the /fs and /include/linux directories. And call them trashbin.[ch] ;) > Also, VFS would require just a couple of trivial changes to support > something like '-o trashbin' mount-time option for all file systems. > In addition, individual file systems may support per-file attributes for > this (e.g., ext2/3/4). Sounds like a good idea to me. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
> > As we promised on the linux-ext4 list on October 31, here is the patch > > that adds secure deletion via a trash-bin functionality for ext4. It is a > > compromise solution that combines secure deletion with the trash-bin support > > (the latter had been requested by even more people than the former :-). > > Given that almost all of the code for this uses vfs interfaces and > only a couple of simple filesystem hooks, is there any reason for > this being ext4 specific? i.e. why aren't you hooking the vfs layer > so we get a single undelete/secure delete implementation for all > filesystems? You are right. Actually, we mentioned it as a benefit number 4 of this approach in the original post. Most of the code is not file-system--specific and can be used by any other (all other?) file system(s). The only complication is that only ext2/3/4 and reiser file systems already support the per-file secure deletion and undelete attributes. Since ext4 is in early development now, we believe it'd be easier to get such code into ext4 than into the main-line VFS. If there's enough interested among the kernel maintainers, we'd be happy to move this functionality to the VFS and provide f/s hooks for secure-deletion/trash-bin. I guess, the right thing to do would be to move the common trash-bin (tb.c and tb.h) code into the /fs and /include/linux directories. Also, VFS would require just a couple of trivial changes to support something like '-o trashbin' mount-time option for all file systems. In addition, individual file systems may support per-file attributes for this (e.g., ext2/3/4). Since I just postponed my moving house (I am joining IBM soon) I think we may have time to try it out this week. Nikolai. --- Filesystems and Storage Laboratory, Stony Brook University - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4
On Mon, Dec 04, 2006 at 01:33:55PM -0500, Nikolai Joukov wrote: > As we promised on the linux-ext4 list on October 31, here is the patch > that adds secure deletion via a trash-bin functionality for ext4. It is a > compromise solution that combines secure deletion with the trash-bin support > (the latter had been requested by even more people than the former :-). Given that almost all of the code for this uses vfs interfaces and only a couple of simple filesystem hooks, is there any reason for this being ext4 specific? i.e. why aren't you hooking the vfs layer so we get a single undelete/secure delete implementation for all filesystems? Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
