Re: Authoring a versioning plugin
Le Fri, 13 Jan 2006 23:17:06 +0200, Toomas Laasik [EMAIL PROTECTED] a écrit : Hello, Here are some links that I have found while searching such file systems that can keep versions or log of changes: While not currently implemented as a filesystem, Subversion repositories look-and-feel is very much like a filesystem. I mention it, because instead of maintaining revisions of each file separately (RCS, then CVS, etc.), it maintains revisions (simple integer counter) of the repository as a whole. It nicely solve the problem of deleted (or simply moved around) files. Tagging versions, creating branches, is handled orthogonally, by copying whole trees in other directories (of course internally everything is just hardlinked). What I mean is, that different approach could give ideas on how to implement versionning at filesystem level, or maybe improve a filesystem as a better backend (or frontend ?) to Subversion... Hope it helps, Pierre.
Re: Authoring a versioning plugin
Peter van Hardenberg wrote:
On January 13, 2006 03:00 pm, you wrote:
On Fri, 13 Jan 2006, Hans Reiser wrote:
If someone figures out why we can't do it but /proc can, or even fixes
it, it would be good.
It wouldn't be something so simple as echo's trailing newline, would it?
-Jonathan
Good thought, but no:
[...]
[EMAIL PROTECTED]:/home/norbs/reiser/emptydir//plugin$ echo ext-1 -n
fibration
[...]
[EMAIL PROTECTED]:/home/norbs/reiser/emptydir//plugin$ irb
irb(main):004:0 f = open('fibration', 'w')
= #File:fibration
irb(main):005:0 f.write('ext-1')
= 5
irb(main):006:0 f.close
= nil
irb(main):007:0 exit
[...]
I'd look more closely at how proc does it and how we do it, but I'm too tired
for source digging tonight.
I think the difference is that Reiser expects a null-terminated string,
whereas Proc doesn't need the null, and can chop off newlines when needed.
So, I think this would also work:
echo -en 'ext-1\0' fibration
Re: Authoring a versioning plugin
David Masover wrote:
Peter van Hardenberg wrote:
On January 13, 2006 03:00 pm, you wrote:
On Fri, 13 Jan 2006, Hans Reiser wrote:
If someone figures out why we can't do it but /proc can, or even fixes
it, it would be good.
It wouldn't be something so simple as echo's trailing newline, would it?
-Jonathan
Good thought, but no:
[...]
[EMAIL PROTECTED]:/home/norbs/reiser/emptydir//plugin$ echo ext-1 -n
fibration
[...]
[EMAIL PROTECTED]:/home/norbs/reiser/emptydir//plugin$ irb
irb(main):004:0 f = open('fibration', 'w')
= #File:fibration
irb(main):005:0 f.write('ext-1')
= 5
irb(main):006:0 f.close
= nil
irb(main):007:0 exit
[...]
I'd look more closely at how proc does it and how we do it, but I'm too tired
for source digging tonight.
I think the difference is that Reiser expects a null-terminated string,
I'll happily take a patch to fix that.
whereas Proc doesn't need the null, and can chop off newlines when needed.
So, I think this would also work:
echo -en 'ext-1\0' fibration
Re: Authoring a versioning plugin
Bedros Hanounik wrote: David, I appreciate your criticism, but we're not in a flame war. I never claimed to be an FS expert. Take it easy; you don't have to beat my suggestion to death. There's no perfect solution, and all feedbacks, no matter how idiotic or simple may seem, help making a better final solution. I didn't mean this to be a personal attack, or a flame. I was trying to be thorough, not confrontational. You did actually have a good idea -- the '.rev' -- but I just wanted to make sure that if it was done, it would be done right. Notice also that my last paragraph on that topic was how I think it should be done right -- which isn't that different than your idea. Again, I'm not the FS expert here. Nor am I. Just a fan, really.
Re: Authoring a versioning plugin
Peter van Hardenberg wrote:
Hi Yoanis, good to see you're still pursuing this.
On January 11, 2006 02:59 pm, Yoanis Gil Delgado wrote:
This are the intentions:
To write a versioning plugin that will allows the file system user to
easily revert the files under versioning to a some previous state. The
plugin will allow to revert the file state, based on revisions number and
date modifications(and not sure about this one). There will be a special
pseudo file named previous that will return the previous version of the
file. The final result should allow to the the following actions:
$ echo 1 myfile.txt (let's say we make this command at Wed Jan 11
16:53:55) $ echo 2 myfile.txt (let's say we make this command at Wed Jan
11 16:54:57) $ echo 3 myfile.txt (let's say we make this command at Wed
Jan 11 16:55:59)
Suppose you want the latest version, then you type:
$ cat myfile/.../previous
Some other content
Or you want the n-th version, then you type:
$ cat myfile/.../1
Some content
$ cat myfile/.../2
Some other content
$ cat myfile/.../3
This is going to clutter the ... directory rather a lot. Instead of adding
more files into (which, by the way, is completely obscure) I would
suggest you create a new pseudo directory.
Perhaps:
$ cat myfile/.^4/history/previous
$ cat myfile/.^4/history/version/1
still not quite right, but at least it contains a bit more information about
what the 1 refers to.
Ok, lets use myfile//versions/1 etc.
I imagine that attribute should be
$ echo 1 myfile.txt//plugins/versioning
or
$ echo everywrite myfile.txt//plugins/versioning
Unfortunately, my experience is that you cannot use echo to change the data
in the plugins/* pseudoplugins, even when it should be legal to do so. I just
had a little ruby script that looked roughly like this:
f = open pseudofile;
f.write('newplugin');
f.close;
Never had the time to figure out why that was necessary, but there it is.
(There is a comment on the plugin-wiki gotchas section.)
If someone figures out why we can't do it but /proc can, or even fixes
it, it would be good.
Re: Authoring a versioning plugin
Yoanis Gil Delgado wrote: On Thursday 12 January 2006 06:56 pm, you wrote: David, I appreciate your criticism, but we're not in a flame war. I never claimed to be an FS expert. Take it easy; you don't have to beat my suggestion to death. There's no perfect solution, and all feedbacks, no matter how idiotic or simple may seem, help making a better final solution. my suggestions were burst of the moment, I didn't give 'em much thoughts; however, all the problems you found could be fixed. Again, I'm not the FS expert here. -B Yes I agree with you Bedros, but i don't think David wanted to beat your suggestion to death. You're suggestions make me thinks things I have not preview. As you say the idea it's to find a good solution. What was it you agreed with?
Re: Authoring a versioning plugin
Toomas Laasik wrote: Hello, Here are some links that I have found while searching such file systems that can keep versions or log of changes: http://logfs.sourceforge.net/ - small logging filesystem project started by a student, but not very stable yet http://wayback.sourceforge.net/ - written in Perl, based on quite old version on FUSE API http://sourceforge.net/projects/cvsfs - uses CVS as file system, based on FUSE API http://www.opensolaris.org/os/community/zfs/ - very powerful file system that supports snapshots of files Microsoft has made something too on their server platform, but I couldn't find a link for that. It seems to me that many have started with it, but haven't finished. Creating a versioning FS that is comercially stable and fast takes a lot of effort and is quite complex. Go discover what and how others have done it and how to do it best in ReiserFS. I would be very interested in stable file system that supports automatic versioning. Maybe ReiserFS becomes the first one that I can actually use. I don't want a userspace FS, but a good FS where I can safely keep my FreeBSD installation. Toomas Clearcase is out there and finished and it works. Several thousand dollars a seat though ;-) Hans
Re: Authoring a versioning plugin
On Wed, 2006-01-11 at 22:44 -0800, Hans Reiser wrote: Hans Reiser wrote: I am skeptical that having it occur with every write is desirable actually. Consider the case where you type cat file1 file2. This will produce a version of file2 for every 4k that is in file1, because (well I didn't look at the bash source, but I would guess) it appends in 4k incremental writes rather than one big write. Versioning on file close makes more sense [snip] Not that my opinion means anything. :-) But I agree with Hans that file close is the place to create the new version. The plugin should track the writes (and mmap flushes) between file open and close, then on file close it can process everything into a reverse binary diff to save permanently. -- Jonathan Briggs [EMAIL PROTECTED] eSoft, Inc. signature.asc Description: This is a digitally signed message part
Re: Authoring a versioning plugin
Hi Yoanis, good to see you're still pursuing this.
On January 11, 2006 02:59 pm, Yoanis Gil Delgado wrote:
This are the intentions:
To write a versioning plugin that will allows the file system user to
easily revert the files under versioning to a some previous state. The
plugin will allow to revert the file state, based on revisions number and
date modifications(and not sure about this one). There will be a special
pseudo file named previous that will return the previous version of the
file. The final result should allow to the the following actions:
$ echo 1 myfile.txt (let's say we make this command at Wed Jan 11
16:53:55) $ echo 2 myfile.txt (let's say we make this command at Wed Jan
11 16:54:57) $ echo 3 myfile.txt (let's say we make this command at Wed
Jan 11 16:55:59)
Suppose you want the latest version, then you type:
$ cat myfile/.../previous
Some other content
Or you want the n-th version, then you type:
$ cat myfile/.../1
Some content
$ cat myfile/.../2
Some other content
$ cat myfile/.../3
This is going to clutter the ... directory rather a lot. Instead of adding
more files into (which, by the way, is completely obscure) I would
suggest you create a new pseudo directory.
Perhaps:
$ cat myfile/.^4/history/previous
$ cat myfile/.^4/history/version/1
still not quite right, but at least it contains a bit more information about
what the 1 refers to.
Some other content
Some more content
$
Or the version nearest to some date, then you type:
$ cat myfile.txt/.../Wed\ Jan\ 11\ 16:50
Some other content
There are already userspace tools which can determine the file creation date.
Just use those, instead of dealing with date parsing in kernel-space. Date
parsing is a way, WAY more subtle problem than you want to deal with. To see
a group that has spent some time on it, check out the Date::Parser for Perl.
Using grep or find, or ls or whatever other tool will accomplish this in
a much more thorough and Unix-consistent way and also save you a pile of
coding time. Believe me, you're going to need it.
Also , there will be an special attribute named under_versioning(or
something like that), that will tell if the file is under versioning. This
plugin will not track directories version, although it's a future plan(I
think this should be mixed with some undelete plugin).
I imagine that attribute should be
$ echo 1 myfile.txt//plugins/versioning
or
$ echo everywrite myfile.txt//plugins/versioning
Unfortunately, my experience is that you cannot use echo to change the data
in the plugins/* pseudoplugins, even when it should be legal to do so. I just
had a little ruby script that looked roughly like this:
f = open pseudofile;
f.write('newplugin');
f.close;
Never had the time to figure out why that was necessary, but there it is.
(There is a comment on the plugin-wiki gotchas section.)
I'm planning to use a delta techniques for versioning storage (delta
compression). The versioning will be at the write level. The versions will
be saved in a special directory under the filesystem. I think the hard part
is the one related to detecting the changes (a COW it's a possible
solution, but i think it's to expensive). I'm thinking a possible solution
will be detecting the bytes changing in each write and archiving then as
the difference. This introduce some problems like :
1-) What happens if the file shrinks?
2-) What happens if the file grows ?
I will send another email with a solution to this problems.
This will not be easy, I look forward to seeing your solution.
I've also plans to extent the documentantion of plugins creation in reiser4
with the experiences of this project. I'll be working in this plugin for
more than 4 months. If you're interested you're welcome to the the
team(just me right now :D )
Well, I have my own fish to fry, but I hope you will document your experiences
on the Reiser4 programmer's wiki, currently housed at:
http://pvh.ca/trac/wiki/reiser4
There is lots of important information there for new Reiser4 plugin
developers, and it will continue to grow as time goes by.
Well... I think this is all (for now :D ). Please let me know what you
think.
I would second Hans' suggestion about a /version/snapshot file which
would essentially act like a cvs commit on that file. I'd suggest that
there be two similar versioning plugins, one which automatically versions
after each write, and one which only does it when explicitly asked to. See
the fibration plugin type for an example of this.
-p
--
Peter van Hardenberg ([EMAIL PROTECTED])
Victoria, BC, Canada
Re: Authoring a versioning plugin
I think versioning plugin is a great idea and I bet there're many people like me waiting for such a plugin. However, I have few questions;what happens when I delete a file? should I loose all history of the file with such action? if there's an undelete plugin, what kind of hooks needed so undelete recovers the full state of the file with history.another concern is backup; if I backup the file or the entire directory (or drive), is it transparent to the backup app, or something extra needed to be done to backup the history of the file? if you store all the history in a sub direcotry let's say .rev and make it generic (and hence visible) to everyone, the above problems will go away. for example filename.ext deltas could be stored in .rev/filename- rev-date-time.delta with base rev in .rev/filename-rev-date-time.extcorrect me if I'm missing something, because I don't know the plugin mechanism of reiser4.-BOn 1/12/06, Jonathan Briggs [EMAIL PROTECTED] wrote: On Wed, 2006-01-11 at 22:44 -0800, Hans Reiser wrote: Hans Reiser wrote: I am skeptical that having it occur with every write is desirable actually. Consider the case where you type cat file1 file2.This will produce a version of file2 for every 4k that is in file1, because (well I didn't look at the bash source, but I would guess) it appends in 4k incremental writes rather than one big write.Versioning on file close makes more sense[snip]Not that my opinion means anything. :-) But I agree with Hans that fileclose is the place to create the new version.The plugin should trackthe writes (and mmap flushes) between file open and close, then on file close it can process everything into a reverse binary diff to savepermanently.--Jonathan Briggs [EMAIL PROTECTED]eSoft, Inc.-BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux)iD8DBQBDxoTGG8fHaOLTWwgRAkE0AJ9qn75iDGZTgO+KpOEcVVqOuT8mgwCcDYyyBHWE+E4oBHkabjZZHNIoAqM==Vtbo-END PGP SIGNATURE-
Re: Authoring a versioning plugin
On Thursday 12 January 2006 02:34 pm, you wrote: On Thursday 12 January 2006 01:44 am, you wrote: Hans Reiser wrote: I am skeptical that having it occur with every write is desirable actually. Consider the case where you type cat file1 file2. This will produce a version of file2 for every 4k that is in file1, because (well I didn't look at the bash source, but I would guess) it appends in 4k incremental writes rather than one big write. Versioning on file close makes more sense, but I suggest manual control using the /checkin pseudofile, and then we can reasonably make it the default plugin for the whole FS (write it so that it calls the other plugins so that when we change the other plugins we don't need to change your code to do it). People who don't want versioning will simply never touch the checkin pseudofile. Make sure that for that case there is just an if statement condition check as overhead, and there will be no reason to not make versioning the default plugin that happens to do nothing unless you use the checkin pseudofile at least once during the life of the file. hmm, maybe /snap is better than /checkin ? Well, let's decide that once the code is written;-) Do you agree with my points here? Yes I agree with your points. Still, i will like that some files have auto versioning.
Re: Authoring a versioning plugin
On Thursday 12 January 2006 03:02 pm, you wrote: Please remember the plugin it's in an earlier design phase, and the answers can change, but right now this is what I think: I think versioning plugin is a great idea and I bet there're many people like me waiting for such a plugin. However, I have few questions; what happens when I delete a file? should I loose all history of the file with such action? I think the history should go away too, since history will be stored as deltas. if there's an undelete plugin, what kind of hooks needed so undelete recovers the full state of the file with history. Undelete plugin is for future work. right now I'm thinking that if there is mechanism to track down the versioning info of a file no matter the directory it's located, then this should not be a problem. another concern is backup; if I backup the file or the entire directory (or drive), is it transparent to the backup app, or something extra needed to be done to backup the history of the file? don't know right now. will come with more answers on next days. if you store all the history in a sub direcotry let's say .rev and make it generic (and hence visible) to everyone, the above problems will go away. for example filename.ext deltas could be stored in .rev/filename- rev-date-time.delta with base rev in .rev/filename-rev-date-time.ext But what happens if i type: $ mv filename.ext ../ then the entire file revision tree must be copied. that's why i mention the idea of mechanism to track down the versioning info of a file. correct me if I'm missing something, because I don't know the plugin mechanism of reiser4. Thanks a lot for the questions.
Re: Authoring a versioning plugin
On Thursday 12 January 2006 02:39 pm, you wrote: On Thursday 12 January 2006 01:14 pm, you wrote: I'm planning to use a delta techniques for versioning storage (delta compression). The versioning will be at the write level. The versions will be saved in a special directory under the filesystem. I think the hard part is the one related to detecting the changes (a COW it's a possible solution, but i think it's to expensive). I'm thinking a possible solution will be detecting the bytes changing in each write and archiving then as the difference. This introduce some problems like : 1-) What happens if the file shrinks? 2-) What happens if the file grows ? I will send another email with a solution to this problems. This will not be easy, I look forward to seeing your solution. Well this is one of the interesting part of the projects. I will not start from scratch, since there is previous work on this area (the delta file system for example, although it's a little old).
Re: Authoring a versioning plugin
On Thu, 2006-01-12 at 15:05 -0500, Yoanis Gil Delgado wrote: On Thursday 12 January 2006 02:34 pm, you wrote: On Thursday 12 January 2006 01:44 am, you wrote: Hans Reiser wrote: I am skeptical that having it occur with every write is desirable actually. Consider the case where you type cat file1 file2. This will produce a version of file2 for every 4k that is in file1, because (well I didn't look at the bash source, but I would guess) it appends in 4k incremental writes rather than one big write. Versioning on file close makes more sense, but I suggest manual control using the /checkin pseudofile, and then we can reasonably make it the default plugin for the whole FS (write it so that it calls the other plugins so that when we change the other plugins we don't need to change your code to do it). People who don't want versioning will simply never touch the checkin pseudofile. Make sure that for that case there is just an if statement condition check as overhead, and there will be no reason to not make versioning the default plugin that happens to do nothing unless you use the checkin pseudofile at least once during the life of the file. hmm, maybe /snap is better than /checkin ? Well, let's decide that once the code is written;-) Do you agree with my points here? Yes I agree with your points. Still, i will like that some files have auto versioning. Would you be able to enable auto versioning for an entire directory, including all new files created in it? For instance I would like to enable auto versioning on the /etc/ directory, so I can always track changes to config files. Also I assume it will track which UID makes the change? -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: Authoring a versioning plugin
Bedros Hanounik wrote: I think versioning plugin is a great idea and I bet there're many people like me waiting for such a plugin. However, I have few questions; what happens when I delete a file? should I loose all history of the file with such action? Depends. Delete has always been a modification of the directory, not the file. So, if you were versioning an entire directory tree (recursively, of course), then deleting a single file within that tree would be a modification to its parent directory. Until the changelog/history/old versions of the parent directory all disappear, there will be a link to the file, meaning the file still behaves exactly as it always did, it's just that instead of only being linked to in foo/bar it's now in foo/.../version/1234/bar [...] if you store all the history in a sub direcotry let's say .rev and make it generic (and hence visible) to everyone, the above problems will go away. Only now you have some new problems: What if someone wants to make a .rev for something else? '...' was carefully selected, and is still debated. Now you're suggesting each plugin be able to have its own file/directory in every normal directory? NO. Will not work. What might work is creating a '.metadata-archive' directory, which has hardlinks to everything tar is allowed to backup from '...' -- that way, we only need to come up with one more unique name than we already have. Technically, not even that -- we could hide a directory with any name we like inside '...', and let tar archive '...', but not, say, '.../notar', but that's a hassle for users. Anyone have a better name for '.metadata-archive', though? You also have another problem: You've just broken 'rm -r' and 'mv' across partitions/devices. Consider: Am I allowed to remove '.metadata-archive'? If yes, then what happens when I look for it again? Is it magically still there? And if no, how am I supposed to ever be able to rmdir a directory that's technically not empty? Should that just magically work? I suspect that we'd be fine if we just say that removing it should pretend to work (return an OK status), and rmdir should actually work, but it should always show up in the directory listing, because I can't think of why a program would step through the directory listing, unlinking each file, and then get another directory listing, to make sure they're all gone, before it tried 'rmdir'-ing the directory. It makes no sense -- you try to 'rmdir', and you complain to the user if directory not empty because someone stuck a new file in the dir while you were trying to delete it. for example filename.ext deltas could be stored in .rev/filename- rev-date-time.delta with base rev in .rev/filename-rev-date-time.ext I don't like your semantics. I just don't. It makes sense to use file extensions for fibration, because you can guess that '.o' files are similar to each other. Same with '.c' files. But fibration is completely transparent, and just affects performance, so it's ok to guess there. When it comes to actually affecting semantics, file extensions have always been irrelevant except to programs that care, and those programs only care about their own extensions. For instance, say I have foo.tar.bz2 Obviously, this way looks bad: foo.tar-rev-date-time.delta foo.tar-rev-date-time.bz2 So do we do it this way? foo-rev-date-time.delta foo-rev-date-time.tar.bz2 No, because then what if you have this? foo.01.12.2006.tar.bz2 Obviously, this is wrong: foo-rev-date-time.delta foo-rev-date-time.01.12.2006.tar.bz2 And this is just as wrong as it was before: foo.01.12.2006.tar-rev-date-time.delta foo.01.12.2006.tar-rev-date-time.bz2 And for that matter, what if I have these? foo.tbz2 foo.zip Both become foo.delta. And just a minor thing, but you're also making the limit on filenames just that much more restrictive. Instead, I suggest we do it this way: foo/.../version/rev with symlinks to some handy versions of foo/.../version/date foo/.../version/time This should also work if foo is a directory. So to actually publish a given revision, you do tar -cjSf foo-1234.tar.bz2 foo/.../1234 Now, what about forking? If we had good copy-on-write support, we could just do something like cp --cow foo/.../1234 foo-fork Obviously, this would use the same underlying mechanism of copy-on-write that is used for the versioning system, so there's no need for the system itself to know about forks. What about merges? I don't know, but my knowledge of existing versioning systems is pretty limited...
Re: Authoring a versioning plugin
Peter van Hardenberg wrote: Hi Yoanis, good to see you're still pursuing this. On January 11, 2006 02:59 pm, Yoanis Gil Delgado wrote: I would second Hans' suggestion about a /version/snapshot file which would essentially act like a cvs commit on that file. I'd suggest that there be two similar versioning plugins, one which automatically versions after each write, and one which only does it when explicitly asked to. See the fibration plugin type for an example of this. Sounds good. I'd propose a third: auto-versioning with optional commits. Every commit nukes all previous auto-verisons and adds a long-term version. That is: The file foo/.../version/1234 would be the version before foo/.../version/auto/1 And if you committed foo/.../version/auto/5678 it would become foo/.../version/1235 and foo/.../version/auto/* would be nuked. That way, you can protect yourself from doing something extremely stupid, such as rm file, without having to go back to a manual version, while at the same time having a sane set of manual versions (where you know you didn't do something *that* stupid) to keep your disk usage sane, and to make it easier to go back and find something that genuinely was a previous version, and not just an oops, the cat stepped on the keyboard and nuked all my changes version.
Re: Authoring a versioning plugin
David,I appreciate your criticism, but we're not in a flame war. I never claimed to be an FS expert. Take it easy; you don't have to beat my suggestion to death. There's no perfect solution, and all feedbacks, no matter how idiotic or simple may seem, help making a better final solution. my suggestions were burst of the moment, I didn't give 'em much thoughts; however, all the problems you found could be fixed. Again, I'm not the FS expert here.-BOn 1/12/06, David Masover [EMAIL PROTECTED] wrote: Bedros Hanounik wrote: I think versioning plugin is a great idea and I bet there're many people like me waiting for such a plugin. However, I have few questions; what happens when I delete a file? should I loose all history of the file with such action?Depends.Delete has always been a modification of the directory, notthe file.So, if you were versioning an entire directory tree(recursively, of course), then deleting a single file within that tree would be a modification to its parent directory.Until thechangelog/history/old versions of the parent directory all disappear,there will be a link to the file, meaning the file still behaves exactlyas it always did, it's just that instead of only being linked to in foo/barit's now infoo/.../version/1234/bar[...] if you store all the history ina sub direcotry let's say .rev and make it generic (and hence visible) to everyone, the above problems will go away.Only now you have some new problems:What if someone wants to make a .rev for something else?'...' wascarefully selected, and is still debated.Now you're suggesting eachplugin be able to have its own file/directory in every normal directory? NO.Will not work.What might work is creating a '.metadata-archive' directory, which hashardlinks to everything tar is allowed to backup from '...' -- that way,we only need to come up with one more unique name than we already have. Technically, not even that -- we could hide a directory with any name welike inside '...', and let tar archive '...', but not, say, '.../notar',but that's a hassle for users.Anyone have a better name for '.metadata-archive', though?You also have another problem:You've just broken 'rm -r' and 'mv'across partitions/devices.Consider:Am I allowed to remove'.metadata-archive'?If yes, then what happens when I look for it again?Is it magically still there?And if no, how am I supposed toever be able to rmdir a directory that's technically not empty?Shouldthat just magically work?I suspect that we'd be fine if we just say that removing it should pretend to work (return an OK status), and rmdir should actually work,but it should always show up in the directory listing, because I can'tthink of why a program would step through the directory listing,unlinking each file, and then get another directory listing, to make sure they're all gone, before it tried 'rmdir'-ing the directory.Itmakes no sense -- you try to 'rmdir', and you complain to the user ifdirectory not empty because someone stuck a new file in the dir while you were trying to delete it. for example filename.ext deltas could be stored in.rev/filename- rev-date-time.delta with base rev in .rev/filename-rev-date-time.extI don't like your semantics.I just don't. It makes sense to use file extensions for fibration, because you canguess that '.o' files are similar to each other.Same with '.c' files.But fibration is completely transparent, and just affects performance, so it's ok to guess there.When it comes to actually affecting semantics, file extensions havealways been irrelevant except to programs that care, and those programsonly care about their own extensions. For instance, say I havefoo.tar.bz2Obviously, this way looks bad:foo.tar-rev-date-time.deltafoo.tar-rev-date-time.bz2So do we do it this way?foo-rev-date-time.deltafoo-rev-date-time.tar.bz2 No, because then what if you have this?foo.01.12.2006.tar.bz2Obviously, this is wrong:foo-rev-date-time.deltafoo-rev-date-time.01.12.2006.tar.bz2And this is just as wrong as it was before: foo.01.12.2006.tar-rev-date-time.deltafoo.01.12.2006.tar-rev-date-time.bz2And for that matter, what if I have these?foo.tbz2foo.zipBoth become foo.delta.And just a minor thing, but you're also making the limit on filenames just that much more restrictive.Instead, I suggest we do it this way:foo/.../version/revwith symlinks to some handy versions offoo/.../version/datefoo/.../version/time This should also work if foo is a directory.So to actually publish agiven revision, you dotar -cjSf foo-1234.tar.bz2 foo/.../1234Now, what about forking?If we had good copy-on-write support, we could just do something likecp --cow foo/.../1234 foo-forkObviously, this would use the same underlying mechanism of copy-on-writethat is used for the versioning system, so there's no need for thesystem itself to know about forks. What about merges?I don't know, but my knowledge of existingversioning systems is pretty limited...
Re: Authoring a versioning plugin
On Thursday 12 January 2006 06:56 pm, you wrote: David, I appreciate your criticism, but we're not in a flame war. I never claimed to be an FS expert. Take it easy; you don't have to beat my suggestion to death. There's no perfect solution, and all feedbacks, no matter how idiotic or simple may seem, help making a better final solution. my suggestions were burst of the moment, I didn't give 'em much thoughts; however, all the problems you found could be fixed. Again, I'm not the FS expert here. -B Yes I agree with you Bedros, but i don't think David wanted to beat your suggestion to death. You're suggestions make me thinks things I have not preview. As you say the idea it's to find a good solution.
Re: Authoring a versioning plugin
Yoanis Gil Delgado wrote: This are the intentions: To write a versioning plugin that will allows the file system user to easily revert the files under versioning to a some previous state. The plugin will allow to revert the file state, based on revisions number and date modifications(and not sure about this one). There will be a special pseudo file named previous that will return the previous version of the file. The final result should allow to the the following actions: $ echo 1 myfile.txt (let's say we make this command at Wed Jan 11 16:53:55) $ echo 2 myfile.txt (let's say we make this command at Wed Jan 11 16:54:57) $ echo 3 myfile.txt (let's say we make this command at Wed Jan 11 16:55:59) Suppose you want the latest version, then you type: $ cat myfile/.../previous not ... ;-) Some other content Or you want the n-th version, then you type: $ cat myfile/.../1 Some content $ cat myfile/.../2 Some other content $ cat myfile/.../3 Some other content Some more content $ Or the version nearest to some date, then you type: $ cat myfile.txt/.../Wed\ Jan\ 11\ 16:50 Some other content Also , there will be an special attribute named under_versioning(or something like that), that will tell if the file is under versioning. This plugin will not track directories version, although it's a future plan(I think this should be mixed with some undelete plugin). How about using myfile//1 ;-) I'm planning to use a delta techniques for versioning storage (delta compression). Ok, good. The versioning will be at the write level. Please make two plugins, one of which is the default, and for which versioning occurs only when you touch myfile//checkin, and the other occurs with each change. I am skeptical that having it occur with every write is desirable actually. The versions will be saved in a special directory under the filesystem. I think the hard part is the one related to detecting the changes (a COW it's a possible solution, but i think it's to expensive). I'm thinking a possible solution will be detecting the bytes changing in each write and archiving then as the difference. This introduce some problems like : 1-) What happens if the file shrinks? 2-) What happens if the file grows ? Please consider that myfile//checkin will be less work to code. I will send another email with a solution to this problems. I've also plans to extent the documentantion of plugins creation in reiser4 with the experiences of this project. I'll be working in this plugin for more than 4 months. If you're interested you're welcome to the the team(just me right now :D ) Well... I think this is all (for now :D ). Please let me know what you think. This all seems great!
Re: Authoring a versioning plugin
Hans Reiser wrote: : I am skeptical that having it occur with every write is desirable actually. Consider the case where you type cat file1 file2. This will produce a version of file2 for every 4k that is in file1, because (well I didn't look at the bash source, but I would guess) it appends in 4k incremental writes rather than one big write. Versioning on file close makes more sense, but I suggest manual control using the /checkin pseudofile, and then we can reasonably make it the default plugin for the whole FS (write it so that it calls the other plugins so that when we change the other plugins we don't need to change your code to do it). People who don't want versioning will simply never touch the checkin pseudofile. Make sure that for that case there is just an if statement condition check as overhead, and there will be no reason to not make versioning the default plugin that happens to do nothing unless you use the checkin pseudofile at least once during the life of the file. hmm, maybe /snap is better than /checkin ? Well, let's decide that once the code is written;-) Do you agree with my points here? Hans
