Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-19 Thread Bill La Forge
One thing I've seen time and again that could easily (a relative term!) be done here is to have an extended bit. Just dedicate one of the currently unused bits to be the extended bit. When set, the 128 byte pointer is actually 256 bytes long. Bill Darren Reed wrote: Eric Schrock wrote: On

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Constantin Gonzalez
Hi, there might be value in a zpool scrub -r (as in re-write blocks) other than the prior discussion on encryption and compression. For instance, a bit that is just about to rot might not be detected with a regular zfs scrub but it would be rewritten with a re-writing scrub. It would also

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Jeff Bonwick
Of course the re-writing must be 100% safe, but that can be done with COW quite easily. Almost. The hard part is snapshots. If you modify a data block, you must also modify every L1 indirect block in every snapshot that points to it, and every L2 above each L1, all the way up to the

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Brian Hechinger
On Tue, Jul 18, 2006 at 09:46:44AM -0400, Chad Mynhier wrote: On 7/18/06, Brian Hechinger [EMAIL PROTECTED] wrote: On Tue, Jul 18, 2006 at 01:27:21AM -0700, Jeff Bonwick wrote: the ability to remap blocks would be *so* useful -- it would enable compression of preexisting data, removing

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Chad Mynhier
On 7/18/06, Brian Hechinger [EMAIL PROTECTED] wrote: On Tue, Jul 18, 2006 at 09:46:44AM -0400, Chad Mynhier wrote: On 7/18/06, Brian Hechinger [EMAIL PROTECTED] wrote: Being able to remove devices from a pool would be a good thing. I can't personally think of any reason that I would ever

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Bill Sommerfeld
On Sun, 2006-07-16 at 15:21, Darren J Moffat wrote: Jeff Victor wrote: Why? Is the 'data is encrypted' flag only stored in filesystem metadata, or is that flag stored in each data block? Like compression and which checksum algorithm it will be stored in every dmu object. I thought

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Bill Moore
On Wed, Jul 19, 2006 at 03:10:00AM +0200, [EMAIL PROTECTED] wrote: So how many of the 128 bits of the blockpointer are used for things other than to point where the block is? 128 *bits*? What filesystem have you been using? :) We've got luxury-class block pointers that are 128 *bytes*. We

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-18 Thread Darren Reed
Bill Moore wrote: On Wed, Jul 19, 2006 at 03:10:00AM +0200, [EMAIL PROTECTED] wrote: So how many of the 128 bits of the blockpointer are used for things other than to point where the block is? 128 *bits*? What filesystem have you been using? :) We've got luxury-class block

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-17 Thread Darren J Moffat
Jeff Victor wrote: Why? Is the 'data is encrypted' flag only stored in filesystem metadata, or is that flag stored in each data block? Like compression and which checksum algorithm it will be stored in every dmu object. If the latter is true, it would be possible (though potentially

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-17 Thread Darren J Moffat
Bill Sommerfeld wrote: On Fri, 2006-07-14 at 07:03, Darren J Moffat wrote: The current plan is that encryption must be turned on when the file system is created and can't be turned on later. This means that the zfs-crypto work depends on the RFE to set properties at file system creation

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-17 Thread Luke Scharf
Darren J Moffat wrote: Buth the reason thing is how do you tell the admin "its done now the filesystem is safe". With compression you don't generally care if some old stuff didn't compress (and with the current implementation it has to compress a certain amount or it gets written uncompressed

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-16 Thread Jeff Victor
Darren J Moffat wrote: Darren Reed wrote: Hmmm, well, I suppose the same problem might apply to encrypting data too...so maybe what I need is a zfs command that will walk the filesystem's data tree, read in data and write it back out according to the current data policy. And if that file

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-16 Thread Joseph Mocker
Jeff Victor wrote: And if that file system is multiple terrabytes would you be okay with there being a read and write lock while this runs ? I am only guessing, but when encryption is important enough the answer is yes. But the next question is then is this situation common enough to

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-14 Thread Darren J Moffat
The problem with encrypting in place is knowing when the system is safe, ie when it is done. There are lots of things in this area that are possible but they certainly aren't in the first deliveries for this project and with all due respect no amount of pushing by anyone will change my mind

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-14 Thread Bill Sommerfeld
On Fri, 2006-07-14 at 07:03, Darren J Moffat wrote: The current plan is that encryption must be turned on when the file system is created and can't be turned on later. This means that the zfs-crypto work depends on the RFE to set properties at file system creation time. You also won't

Re: [zfs-discuss] Enabling compression/encryption on a populated filesystem

2006-07-13 Thread Chad Mynhier
On 7/13/06, Darren Reed [EMAIL PROTECTED] wrote: When ZFS compression is enabled, although the man page doesn't explicitly say this, my guess is that only new data that gets written out is compressed - in keeping with the COW policy. [ ... ] Hmmm, well, I suppose the same problem might apply