Re: future r4 maintenance question
Maciej Sołtysiak wrote: Hello Hans, Saturday, July 22, 2006, 8:03:28 PM, you wrote: We are going to give changing the paradigm a try. The difference between 4.1-beta and 4.0 is that different plugins are the default, and the experimental code is in the plugins you see when mounting with the mount option 4.1-beta. Let's see if it works in practice. I Understand. This is good news. Hm, do you think that reiser4's pluggability is enough to have this single kernel tree (fs/reiser4) for a longer period of time. Yes, reiser4 will have a much longer lifetime, and improvements will come out in small pieces rather than complete rewrites. I think we can add the enhanced semantics one feature at a time. It was the storage layer that was the big thing that had to be right before the rest could proceed. Now, what remains are a whole lot of incremental improvements I hope. If I can make enough money off the repacker or I get funding from France or some other government and we are able to afford to keep vs and zam working on the storage layer, and Nate working on VFS changes and Peter Foldiak and others at St. Andrews doing semantic enhancements and Gorazd doing user space browsers things could get very interesting. We need to get into the kernel, and money will manifest, and programmers can get to work I mean, can you predict a need of spawning something like reiser5 in the forseeable future or would fs/reiser4 + plugins be enough to do away with the future vision and other future * stuff you've written about ? eg. I remember reading about very granular security ACLs like restricting a certain line in a file (like /etc/passwd) Yes, I still want to do that stuff Hans
Re: future r4 maintenance question
I am going to do the enhanced semantics first, so that somebody does not beat me to it. David's examples are good. There's another note to kernel developers -- if Reiser5, 6, and 7 are implemented as suites of plugins on top of Reiser4, then the Reiser4 code will be maintained for a very long time. Kind of like ext2 vs ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with additional mount options. There is definitely a lot that can be done to move Reiser4 (as it is today) closer to the Reiser4 whitepaper on the homepage. ACLs are one thing, files as a directory are another. The idea of v4 is to do away with many cases where a separate namespace is created for no good reason -- for instance, where is the data in an id3 tag? It's inside an mp3 file, and you can only get to it with tools written for id3 tags of mp3 files. The Reiser4 concept is to allow things like that to exist, but not require programs to know about libid3 or whatever. Want to know what the artist of a particular file is? foo.mp3/.../id3/artist Or maybe a more generic way: foo.mp3/.../song-info/artist That way, you could have tools which don't even have to know if the file has an id3 tag, or something entirely new, or if the metadata is being stored outside the main file. It'd be entirely possible to allow that file to be treated as a separate file entirely by the plugin, rather than something derived from foo.mp3. The advantages don't seem immediately obvious until you consider that the program which does this doesn't have to even know that it's dealing with song metadata. Consider some of the one-line shell scripts possible: # Change the artist name for all songs in the directory: for i in *; do echo 'Jimi Hendrix' foo.mp3/.../song-info/artist; done # Make a playlist of all files by Hendrix, mp3 or otherwise: for i in `find`; do if [ `cat $i/.../song-info/artist` == 'Jimi Hendrix' ]; then echo ../$i playlists/hendrix.m3u; fi; done # Copy all files needed by said playlist to a USB device: cp playlist/hendrix.m3u/.../files/* /mnt/usb cp playlist/hendrix.m3u /mnt/usb I'm sure others can think of much more interesting examples. All of that is planned for v4, eventually. It's very pluggable. Well, I think it is. I don't work here...
Re: future r4 maintenance question
On Sat, 2006-07-22 at 14:42 -0500, David Masover wrote: Mike Benoit wrote: [...] You would hope. The problem is, most of the ideas we have for amazing/useless/practical plugins have already been done as specialized filesystems in FUSE. What's more, FUSE doesn't usually require root privileges to run, causing some people to do such useless things as support for various filesystems already in the kernel (so you can loop-mount filesystems without using the loop device or having root privileges). Although, there is one other thing to consider -- NTFS in the kernel has stagnated for something like 10 years without write support, while userspace tools like ntfsck and resize_ntfs, using libntfs, have actually gotten relatively stable at dealing with all kinds of ntfs oddities. For awhile, we had captive-ntfs (also done in FUSE), but more recently, someone wrote a FUSE driver for libntfs, making the first reasonably stable fully featured read/write NTFS driver for Linux. It's also reasonably fast, sometimes faster than ext2/3, I'm told. I am using the official libntfs fuse module but it is not all that fast if you compare how much more cpu power is needed. And even then it is NOT faster then ext2/3. I am copying a couple of 650mb iso's and I have 100% cpu utilisation and it still takes tens of minutes to copy just 1. Fuse is nice for things like gmailfs or beaglefs but if you really want performance, kernelspace imho is the only place to implement a filesystems. Greets Sander
future r4 maintenance question
Hello, one of the concerns of LKML folks about r4 is that namesys after finishing their work with r4 will rush off to reiser5 leaving r4 as stable in a bugfix-mode only I understand that way of things - you don't mess with a stable branch software. But a question arises. Will namesys keep r4 in a bugfix-only mode (as with reiserfs v3) or will it have resources planned not only for bugfixes but for updates too ? I think that's a thing that needs to be very clear. Best regards, Maciej
Re: future r4 maintenance question
On Sat, 2006-07-22 at 17:32 +0200, Pysiak Satriani wrote: Hello, one of the concerns of LKML folks about r4 is that namesys after finishing their work with r4 will rush off to reiser5 leaving r4 as stable in a bugfix-mode only I understand that way of things - you don't mess with a stable branch software. But a question arises. Will namesys keep r4 in a bugfix-only mode (as with reiserfs v3) or will it have resources planned not only for bugfixes but for updates too ? I think that's a thing that needs to be very clear. I don't understand the concerns here? Kernel v2.4 is essentially in bug-fix only mode right now, yet I don't see many people complaining that the kernel developers have abandoned it. I personally think its crazy to expect major features be back ported to old versions. For as long as I've been alive programmers have released a major version of their software, and only maintained it with minor version bug fixes while they work on the next major version to introduce features to it. Hans himself has pretty much followed that same path with ReiserFS V3, and now V4. There just happened to be a several year gap in between. I don't know what Hans has in mind in regards to ReiserV5, but ReiserV4 was designed with plugins in mind so adding additional features (even advanced ones) to it should be much easier than most other file systems. Just like many other applications that support plugins/extensions/addons ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised if Reiser4 doesn't spawn a large community of people completely independent of Namesys that create plugins from the useless, to the amazingly useful for years to come. -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: future r4 maintenance question
We are going to give changing the paradigm a try. The difference between 4.1-beta and 4.0 is that different plugins are the default, and the experimental code is in the plugins you see when mounting with the mount option 4.1-beta. Let's see if it works in practice. Hans Pysiak Satriani wrote: Hello, one of the concerns of LKML folks about r4 is that namesys after finishing their work with r4 will rush off to reiser5 leaving r4 as stable in a bugfix-mode only I understand that way of things - you don't mess with a stable branch software. But a question arises. Will namesys keep r4 in a bugfix-only mode (as with reiserfs v3) or will it have resources planned not only for bugfixes but for updates too ? I think that's a thing that needs to be very clear. Best regards, Maciej
Re: future r4 maintenance question
Just like many other applications that support plugins/extensions/addons ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised if Reiser4 doesn't spawn a large community of people completely independent of Namesys that create plugins from the useless, to the amazingly useful for years to come. if it ever gets finished ;) :)
Re: future r4 maintenance question
Pysiak Satriani wrote: Hello, one of the concerns of LKML folks about r4 is that namesys after finishing their work with r4 will rush off to reiser5 leaving r4 as stable in a bugfix-mode only But a question arises. Will namesys keep r4 in a bugfix-only mode (as with reiserfs v3) or will it have resources planned not only for bugfixes but for updates too ? As I understand it, r4 is currently not finished, especially comparing the whitepaper with reality. So, while the features that are finished are mostly stable, I can't imagine them rushing off to v5 without finishing the features originally planned for v4. Examples would be pseudofiles, custom plugins (per-file and per-directory, also), a repacker, fsync optimization, a crypto/compression plugin... Ultimately, this may happen, but by then, I expect v4 would be fairly stable and complete. Besides, bugfix-only does imply bugfixes, which is about all most other filesystems get these days... Disclaimer: I don't work here. If you want a definitive answer, wait for someone at Namesys to confirm or deny what I've just said.
Re: future r4 maintenance question
Mike Benoit wrote: Just like many other applications that support plugins/extensions/addons ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised if Reiser4 doesn't spawn a large community of people completely independent of Namesys that create plugins from the useless, to the amazingly useful for years to come. You would hope. The problem is, most of the ideas we have for amazing/useless/practical plugins have already been done as specialized filesystems in FUSE. What's more, FUSE doesn't usually require root privileges to run, causing some people to do such useless things as support for various filesystems already in the kernel (so you can loop-mount filesystems without using the loop device or having root privileges). Although, there is one other thing to consider -- NTFS in the kernel has stagnated for something like 10 years without write support, while userspace tools like ntfsck and resize_ntfs, using libntfs, have actually gotten relatively stable at dealing with all kinds of ntfs oddities. For awhile, we had captive-ntfs (also done in FUSE), but more recently, someone wrote a FUSE driver for libntfs, making the first reasonably stable fully featured read/write NTFS driver for Linux. It's also reasonably fast, sometimes faster than ext2/3, I'm told. The big advantage of that is, FUSE is easier to deal with than the kernel, and userland daemons can crash, but if something in the kernel crashes, there goes your OS. Also, userland just has more options. Some FUSE drivers are written in Perl, others in Ruby. I realize there are some things that are really better done in the kernel, like crypto/compression support, and Reiser4 will support things that FUSE currently doesn't do very well. But if we're to pull any filesystem hackers away from FUSE, we need to provide as good or better an API, both in-kernel, and for userspace daemons.
Re: future r4 maintenance question
Hello Hans, Saturday, July 22, 2006, 8:03:28 PM, you wrote: We are going to give changing the paradigm a try. The difference between 4.1-beta and 4.0 is that different plugins are the default, and the experimental code is in the plugins you see when mounting with the mount option 4.1-beta. Let's see if it works in practice. I Understand. This is good news. Hm, do you think that reiser4's pluggability is enough to have this single kernel tree (fs/reiser4) for a longer period of time. I mean, can you predict a need of spawning something like reiser5 in the forseeable future or would fs/reiser4 + plugins be enough to do away with the future vision and other future * stuff you've written about ? eg. I remember reading about very granular security ACLs like restricting a certain line in a file (like /etc/passwd) -- Best regards, Maciej
Re: future r4 maintenance question
Hello Mike, Saturday, July 22, 2006, 8:53:06 PM, you wrote: I don't understand the concerns here? Kernel v2.4 is essentially in bug-fix only mode right now, yet I don't see many people complaining that the kernel developers have abandoned it. I personally think its crazy to expect major features be back ported to old versions. For as long as I've been alive programmers have released a major version of their software, and only maintained it with minor version bug fixes while they work on the next major version to introduce features to it. Hans himself has pretty much followed that same path with ReiserFS V3, and now V4. There just happened to be a several year gap in between. Well, that's completely clear. However, reiserfs has had a different situation: during the bugfix mode, important functionality was beeing added to other filesystems: extended attribute support, data=ordered, data=journaled. Reiserfs v3 had noone but Chris Mason (outside of namesys) to do it. The issue is that reiser4 users should never again feel abandoned like they did with reiserfs v3. Well, actually a lot of people did not feel it or they hadn't known about the state of things. Note that back then reiser4 was basically vapourware if I may say so. Concluding, I am not here to point anything bad, I am looking adoption-stoppers for reiser4. BTW, have Mark Shuttleworth and Hans had a talk about support/sponsorship/ubuntu endorsement/anything? (i know it may be absolutely private stuff, sorry for asking if this is the case) -- Best regards, Maciej
Re: future r4 maintenance question
Maciej Sołtysiak wrote: I Understand. This is good news. Hm, do you think that reiser4's pluggability is enough to have this single kernel tree (fs/reiser4) for a longer period away with the future vision and other future * stuff you've written about ? eg. I remember reading about very granular security ACLs like restricting a certain line in a file (like /etc/passwd) I think the granular security ACLs were always intended to be a v4 thing (with plugins). I believe the Future Vision stuff is about even more namespace unification. I seem to remember v5 was semantic changes and v6 was a semi-distributed FS, but I could have that backwards... Anyway, there are definitely plans for things which are non-trivially beyond the plugin capabilities of Reiser4, though they will almost certainly use Reiser4 as a starting point. There's another note to kernel developers -- if Reiser5, 6, and 7 are implemented as suites of plugins on top of Reiser4, then the Reiser4 code will be maintained for a very long time. Kind of like ext2 vs ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with additional mount options. There is definitely a lot that can be done to move Reiser4 (as it is today) closer to the Reiser4 whitepaper on the homepage. ACLs are one thing, files as a directory are another. The idea of v4 is to do away with many cases where a separate namespace is created for no good reason -- for instance, where is the data in an id3 tag? It's inside an mp3 file, and you can only get to it with tools written for id3 tags of mp3 files. The Reiser4 concept is to allow things like that to exist, but not require programs to know about libid3 or whatever. Want to know what the artist of a particular file is? foo.mp3/.../id3/artist Or maybe a more generic way: foo.mp3/.../song-info/artist That way, you could have tools which don't even have to know if the file has an id3 tag, or something entirely new, or if the metadata is being stored outside the main file. It'd be entirely possible to allow that file to be treated as a separate file entirely by the plugin, rather than something derived from foo.mp3. The advantages don't seem immediately obvious until you consider that the program which does this doesn't have to even know that it's dealing with song metadata. Consider some of the one-line shell scripts possible: # Change the artist name for all songs in the directory: for i in *; do echo 'Jimi Hendrix' foo.mp3/.../song-info/artist; done # Make a playlist of all files by Hendrix, mp3 or otherwise: for i in `find`; do if [ `cat $i/.../song-info/artist` == 'Jimi Hendrix' ]; then echo ../$i playlists/hendrix.m3u; fi; done # Copy all files needed by said playlist to a USB device: cp playlist/hendrix.m3u/.../files/* /mnt/usb cp playlist/hendrix.m3u /mnt/usb I'm sure others can think of much more interesting examples. All of that is planned for v4, eventually. It's very pluggable. Well, I think it is. I don't work here...
Re: future r4 maintenance question
Hello David, Saturday, July 22, 2006, 10:16:48 PM, you wrote: There's another note to kernel developers -- if Reiser5, 6, and 7 are implemented as suites of plugins on top of Reiser4, then the Reiser4 code will be maintained for a very long time. Kind of like ext2 vs ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with additional mount options. This the exact information I was hoping to get. I think it's important for people to realize it. Thanks. # Make a playlist of all files by Hendrix, mp3 or otherwise: for i in `find`; do if [ `cat $i/.../song-info/artist` == 'Jimi Hendrix' ]; then echo ../$i playlists/hendrix.m3u; fi; done I've read about this :-) Are plugins for such a thing entirely/mostly kernel-space? I mean, the power of FUSE is being in user-space... If there was an API to write a id3 plugin it propably be more beneficial. Well, it's plugins. If someone writes a kernelspace id3 plugin, someone else could write a kernelspace plugin with a userspace supplied configuration. -- Best regards, Maciej
