Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-08 Thread Nikolai Joukov
 They are defined but unused in 2.6.19, right? I can't see anywhere
 in the 2.6.19 ext2/3/4/reiser trees that actually those flags,
 including setting and retrieving them from disk. JFS i can see
 sets, clears and retreives them, but not the fielsystems you
 mention. Though I might just be blind. ;)

Well, at least Ext2/3/4 and JFS do store these attributes and retrieve
them from the disk.  However, not a single one of these file systems
supports the related functionality (secure deletion or a trash-bin).
Clearly, this is confusing for the users.

 If all we need to add to XFS is support for those flags, then XFS
 support would be trivial to add.

 Oh, damn. I take that back. We're almost out of flag space in the on
 disk inode - these two flags would use the last 2 flag bits so this
 may require an on disk inode format change in XFS. This will be
 a little more complex than I first thought, but not impossible
 as we already support two on-disk inode format versions.

If you do not see any obvious use for these two bits in the near future
and ready to sacrifice them (or at least one of them) for secure deletion
and trash-bin attributes we would be more than happy to add the actual
functionality to XFS as well.

 I'd certainly like to see new functionality like this added at the
 VFS rather than in any particular filesystem. Don't know about
 anyone else, though

  I guess, the right thing to do would be to move the common trash-bin
  (tb.c and tb.h) code into the /fs and /include/linux directories.

 And call them trashbin.[ch] ;)

In fact, we have already done this :-)

  Also, VFS would require just a couple of trivial changes to support
  something like '-o trashbin' mount-time option for all file systems.
  In addition, individual file systems may support per-file attributes for
  this (e.g., ext2/3/4).

 Sounds like a good idea to me.

Thank you!  So we will submit the corresponding patches soon.  We will
start from the trashbin.c and patches for the individual file systems.  A
little later we will submit a VFS patch to add support for the
'-o trashbin' mount option.

Nikolai Joukov
--
Filesystems and Storage Laboratory
Stony Brook University
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread David Chinner
On Tue, Dec 05, 2006 at 11:41:28AM -0500, Nikolai Joukov wrote:
   As we promised on the linux-ext4 list on October 31, here is the patch
   that adds secure deletion via a trash-bin functionality for ext4.  It is a
   compromise solution that combines secure deletion with the trash-bin 
   support
   (the latter had been requested by even more people than the former :-).
 
  Given that almost all of the code for this uses vfs interfaces and
  only a couple of simple filesystem hooks, is there any reason for
  this being ext4 specific?  i.e. why aren't you hooking the vfs layer
  so we get a single undelete/secure delete implementation for all
  filesystems?
 
 You are right.  Actually, we mentioned it as a benefit number 4 of this
 approach in the original post.  Most of the code is not
 file-system--specific and can be used by any other (all other?) file
 system(s).  The only complication is that only ext2/3/4 and reiser file
 systems already support the per-file secure deletion and undelete
 attributes.

They are defined but unused in 2.6.19, right? I can't see anywhere
in the 2.6.19 ext2/3/4/reiser trees that actually those flags,
including setting and retrieving them from disk. JFS i can see
sets, clears and retreives them, but not the fielsystems you
mention. Though I might just be blind. ;)

If all we need to add to XFS is support for those flags, then XFS
support would be trivial to add.

Oh, damn. I take that back. We're almost out of flag space in the on
disk inode - these two flags would use the last 2 flag bits so this
may require an on disk inode format change in XFS. This will be
a little more complex than I first thought, but not impossible
as we already support two on-disk inode format versions.

 Since ext4 is in early development now, we believe it'd be easier to get
 such code into ext4 than into the main-line VFS.  If there's enough
 interested among the kernel maintainers, we'd be happy to move this
 functionality to the VFS and provide f/s hooks for
 secure-deletion/trash-bin.

I'd certainly like to see new functionality like this added at the
VFS rather than in any particular filesystem. Don't know about
anyone else, though

 I guess, the right thing to do would be to move the common trash-bin
 (tb.c and tb.h) code into the /fs and /include/linux directories.

And call them trashbin.[ch] ;)

 Also, VFS would require just a couple of trivial changes to support
 something like '-o trashbin' mount-time option for all file systems.
 In addition, individual file systems may support per-file attributes for
 this (e.g., ext2/3/4).

Sounds like a good idea to me.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread Josef Sipek
On Wed, Dec 06, 2006 at 08:11:00PM +1100, David Chinner wrote:
 They are defined but unused in 2.6.19, right? I can't see anywhere
 in the 2.6.19 ext2/3/4/reiser trees that actually those flags,
 including setting and retrieving them from disk. JFS i can see
 sets, clears and retreives them, but not the fielsystems you
 mention. Though I might just be blind. ;)
 
 If all we need to add to XFS is support for those flags, then XFS
 support would be trivial to add.
 
 Oh, damn. I take that back. We're almost out of flag space in the on
 disk inode - these two flags would use the last 2 flag bits so this
 may require an on disk inode format change in XFS. This will be
 a little more complex than I first thought, but not impossible
 as we already support two on-disk inode format versions.
 
Hrm. I was toying around with the idea of using a flag to mark inodes as
whiteouts (similar to what BSD does) for Unionfs. I remember that Jan Blunck
tried similar thing in his implementation of VFS unionfs mounts.

I am not entirely convinced that whiteout inode flag is the right way to do
things, but I'm just raising this now as I wouldn't want to wait for new
ondisk format for XFS to say that Unionfs supports XFS. (Assuming that it is
the right approach.) ;-)
 
Josef Jeff Sipek.

-- 
I already backed up the box once, I can do it again!
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread David Chinner
On Wed, Dec 06, 2006 at 07:56:19PM -0500, Josef Sipek wrote:
 On Wed, Dec 06, 2006 at 08:11:00PM +1100, David Chinner wrote:
  They are defined but unused in 2.6.19, right? I can't see anywhere
  in the 2.6.19 ext2/3/4/reiser trees that actually those flags,
  including setting and retrieving them from disk. JFS i can see
  sets, clears and retreives them, but not the fielsystems you
  mention. Though I might just be blind. ;)
  
  If all we need to add to XFS is support for those flags, then XFS
  support would be trivial to add.
  
  Oh, damn. I take that back. We're almost out of flag space in the on
  disk inode - these two flags would use the last 2 flag bits so this
  may require an on disk inode format change in XFS. This will be
  a little more complex than I first thought, but not impossible
  as we already support two on-disk inode format versions.
  
 Hrm. I was toying around with the idea of using a flag to mark inodes as
 whiteouts (similar to what BSD does) for Unionfs. I remember that Jan Blunck
 tried similar thing in his implementation of VFS unionfs mounts.
 
 I am not entirely convinced that whiteout inode flag is the right way to do
 things, but I'm just raising this now as I wouldn't want to wait for new
 ondisk format for XFS to say that Unionfs supports XFS. (Assuming that it is
 the right approach.) ;-)

Maybe we should be using EAs for this sort of thing instead of flags
on the inode? If we keep adding inode flags for generic features
then we are going to force more than just XFS into inode format
changes eventually

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread Josef Sipek
On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote:
 Maybe we should be using EAs for this sort of thing instead of flags
 on the inode? If we keep adding inode flags for generic features
 then we are going to force more than just XFS into inode format
 changes eventually

Aren't EAs slow? Maybe not on XFS but on other filesystems...

Josef Jeff Sipek.

-- 
Linux, n.:
  Generous programmers from around the world all join forces to help
  you shoot yourself in the foot for free. 
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread David Chinner
On Wed, Dec 06, 2006 at 09:35:30PM -0500, Josef Sipek wrote:
 On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote:
  Maybe we should be using EAs for this sort of thing instead of flags
  on the inode? If we keep adding inode flags for generic features
  then we are going to force more than just XFS into inode format
  changes eventually
 
 Aren't EAs slow? Maybe not on XFS but on other filesystems...

Only when they don't fit in the inode itself and extra
disk seeks are needed to retrieve them.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread Nicholas Miell
On Thu, 2006-12-07 at 13:49 +1100, David Chinner wrote:
 On Wed, Dec 06, 2006 at 09:35:30PM -0500, Josef Sipek wrote:
  On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote:
   Maybe we should be using EAs for this sort of thing instead of flags
   on the inode? If we keep adding inode flags for generic features
   then we are going to force more than just XFS into inode format
   changes eventually
  
  Aren't EAs slow? Maybe not on XFS but on other filesystems...
 
 Only when they don't fit in the inode itself and extra
 disk seeks are needed to retrieve them.
 
 Cheers,
 
 Dave.

Also keep in mind that the EA doesn't actually have to have a physical
representation on disk (or, rather, it does, but it doesn't need to be
the same representation used by EAs in the user namespace).

This means that if one of those slow EA filesystems still has room for
flags in the inode, it can synthesize the EA on demand.

This is even preferable to ioctls for the interface to new filesystem
metadata -- if a backup or archive program knows how to store EAs, it
will be able to save and restore any new exotic metadata without any
extra effort.

-- 
Nicholas Miell [EMAIL PROTECTED]

-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread Nathan Scott
On Wed, 2006-12-06 at 20:11 +1100, David Chinner wrote:
 ...
 If all we need to add to XFS is support for those flags, then XFS
 support would be trivial to add.
 
 Oh, damn. I take that back. We're almost out of flag space in the on
 disk inode - these two flags would use the last 2 flag bits so this
 may require an on disk inode format change in XFS. This will be
 a little more complex than I first thought, ...

It should be OK - you can do it without an inode version revision
if you take a second 16 bits for di_flags2 from here...

xfs_dinode_core {
...
__uint8_t   di_pad[8];  /* unused, zeroed space */

Its guaranteed zeroed initially (i.e. all flags unset) and the XFS
get/set flags APIs are 32 bits, so you should be OK there.

Also, it may also be possible to reclaim di_onlink at some point (maybe
now, since 16 bits would be good here) if mkfs.xfs is changed to always
create v2 inodes (dynamic conversion ATM IIRC)... not 100% sure though,
needs more code analysis.

cheers.

-- 
Nathan

-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread Josef Sipek
On Wed, Dec 06, 2006 at 07:14:58PM -0800, Nicholas Miell wrote:
 On Thu, 2006-12-07 at 13:49 +1100, David Chinner wrote:
  On Wed, Dec 06, 2006 at 09:35:30PM -0500, Josef Sipek wrote:
   On Thu, Dec 07, 2006 at 12:44:27PM +1100, David Chinner wrote:
Maybe we should be using EAs for this sort of thing instead of flags
on the inode? If we keep adding inode flags for generic features
then we are going to force more than just XFS into inode format
changes eventually
   
   Aren't EAs slow? Maybe not on XFS but on other filesystems...
  
  Only when they don't fit in the inode itself and extra
  disk seeks are needed to retrieve them.
  
  Cheers,
  
  Dave.
 
 Also keep in mind that the EA doesn't actually have to have a physical
 representation on disk (or, rather, it does, but it doesn't need to be
 the same representation used by EAs in the user namespace).
 
 This means that if one of those slow EA filesystems still has room for
 flags in the inode, it can synthesize the EA on demand.
 
Interesting point. Filesystems such as XFS would be unaffected (at least
with attr v1, not sure how attr2 does things in XFS wrt EAs.)

Another question is, suppose that filesystem x is one of the slow EA
filesystems, exposing this whiteout inode flag as EA would require fs x to
be compiled with EA support. As a matter of fact, it would require all the
filesystems to have EA support. Sure, overall it is less work for me, but I
am more interested in doing the Right Thing(tm) - which may be something
completely different from EA or inode flag (e.g., what Ted Tso suggested at
OLS - effectively a metadata-only ondisk format for unionfs.)
 
 This is even preferable to ioctls for the interface to new filesystem
 metadata -- if a backup or archive program knows how to store EAs, it
 will be able to save and restore any new exotic metadata without any
 extra effort.

Agreed.

Josef Jeff Sipek.

-- 
UNIX is user-friendly ... it's just selective about who it's friends are
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-06 Thread Nathan Scott
On Thu, 2006-12-07 at 12:44 +1100, David Chinner wrote:
 Maybe we should be using EAs for this sort of thing instead of flags
 on the inode? If we keep adding inode flags for generic features
 then we are going to force more than just XFS into inode format
 changes eventually

You do need to be judicious in what you add, but we got up to here
(well over 10 years) with 16 bits worth of space - increasing to 32
bits is going to last a fairly long time... I wouldn't switch to a
different scheme until that point (which is also the point where the
system calls are going to have to change anyway).

cheers.

-- 
Nathan

-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC][PATCH] Secure Deletion and Trash-Bin Support for Ext4

2006-12-04 Thread David Chinner
On Mon, Dec 04, 2006 at 01:33:55PM -0500, Nikolai Joukov wrote:
 As we promised on the linux-ext4 list on October 31, here is the patch
 that adds secure deletion via a trash-bin functionality for ext4.  It is a
 compromise solution that combines secure deletion with the trash-bin support
 (the latter had been requested by even more people than the former :-).

Given that almost all of the code for this uses vfs interfaces and
only a couple of simple filesystem hooks, is there any reason for
this being ext4 specific?  i.e. why aren't you hooking the vfs layer
so we get a single undelete/secure delete implementation for all
filesystems?

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
-
To unsubscribe from this list: send the line unsubscribe linux-fsdevel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html