Re: possible recursive locking detected - while running fs operations in loops - 2.6.18-rc2-git5
On 30/07/06, Hans Reiser <[EMAIL PROTECTED]> wrote: Jesper Juhl wrote: > > Thanks. That's a nice little test suite. > Yes, it is quite useful, our developers have added it to the regression suite That's nice. Now how about that lock validator message I managed to tease out? Akpm said "... the reiserfs locking appears to be unneeded - this inode is going down and nobody else can look it up, so what is to be locked against?" - can you comment on that? -- Jesper Juhl <[EMAIL PROTECTED]> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html
Re: possible recursive locking detected - while running fs operations in loops - 2.6.18-rc2-git5
Jesper Juhl wrote: > > Thanks. That's a nice little test suite. > Yes, it is quite useful, our developers have added it to the regression suite Hans
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
David Masover wrote: >Nikita Danilov wrote: > > > >>As you see, ext2 code already has multiple file "plugins", with >>persistent "plugin id" (stored in i_mode field of on-disk struct >>ext2_inode). >> >> > >Aha! So here's another question: Is it fair to ask Reiser4 to make its >plugins generic, or should we be asking ext2/3 first? > > > Sh., ext* already made their plugins generic, job is done:)
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Nikita Danilov wrote: >Hans Reiser writes: > > David Masover wrote: > > > > > > > > If indeed it can be changed easily at all. I think the burden is on > > > you to prove that you can change it to be more generic, rather than > > > saying "Well, we could do it later, if people want us to..." > > > > None of the filesystems other than reiser4 have any interest in using > > plugins, and this whole argument over how it should be in VFS is > > nonsensical because nobody but us has any interest in using the > > functionality. The burden is on the generic code authors to prove that > > they will ever ever do anything at all besides complain. Frankly, I > > don't think they will. I think they will never produce one line of code. > > > > Please cite one ext3 developer who is signed up to implement ext3 using > > plugins if they are supported by VFS. > >In fact, they all do: > >struct inode_operations ext2_file_inode_operations; >struct inode_operations ext2_dir_inode_operations; >struct inode_operations ext2_special_inode_operations; >struct inode_operations ext2_symlink_inode_operations; >struct inode_operations ext2_fast_symlink_inode_operations; > >As you see, ext2 code already has multiple file "plugins", with >persistent "plugin id" (stored in i_mode field of on-disk struct >ext2_inode). > > > > > Hans > > > >Nikita. > > > > > So the job is already done. Good. Reiser4 can be included then.:) Hans "The Easily Agreeable" Reiser
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Hello David, Saturday, July 29, 2006, 8:11:11 PM, you wrote: > What's more, many distros patch their kernels extensively. They listen > to their users, too. So if there are a lot of users wanting this to be > in the kernel, let them complain -- loudly -- to their distro to patch > for Reiser4. Hmm, what about linspire / freespire ? Linsire is a proud reiser4 debugging sponsor as the website (http://www.namesys.com) says. Wouldn't they want to include reiser4 in their distro first? I removed the busy people from CC to save their time reading this. -- Best regards, Maciej
Re: possible recursive locking detected - while running fs operations in loops - 2.6.18-rc2-git5
On 26/07/06, Andreas Dilger <[EMAIL PROTECTED]> wrote: On Jul 26, 2006 00:16 +0200, Jesper Juhl wrote: > What I did to provoke it was to run 6 different xterms (with a bash > shell) with the following loops in them in a test directory that was > initially empty : > > xterm1: while true; do mkdir a; done > xterm2: while true; do rmdir a; done > xterm3: while true; do touch a/foo; done > xterm4: while true; do find .; done > xterm5: while true; do sync; sleep 1; done > xterm6: while true; do rm -r a; done See racer test at ftp.lustre.org/pub/benchmarks/racer-lustre.tar.gz It does the above, but a bunch more things and is a truly pathalogical test script that does lots of "stupid user tricks", unlike normal tests which are only doing operations that expect to be successful. PS - during the racer.sh test run "rm" is known to segfault after hitting an internal assertion, nobody is sure why. PPS- I don't know who wrote this program, it was originally posted by someone not the author to linux-fsdevel or something. Thanks. That's a nice little test suite. -- Jesper Juhl <[EMAIL PROTECTED]> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Sarath Menon wrote: On Saturday 29 July 2006 23:41, David Masover wrote: I know Gentoo handles this automatically (emerge nvidia-kernel). I hate to say this again, but its not automatically. It requires more My point is, there's a fairly large group of users who would be willing to do that, as they're willing to do that to get their video drivers working. Also, assuming a distro did choose to support it, the only reason nvidia-kernel isn't just distributed with a pre-built kernel (on pre-built OSes, anyway) is licensing. This isn't a problem for Reiser4, which is GPL'd. I suspect that, all technical, political, and "mine is bigger" arguments aside, being available as a root FS of a distro, especially a default FS, would go a long way towards inclusion in the kernel. So all you have to do is find a reasonably popular and friendly distro, with people who are (for the moment) easier to deal with than kernel people. Its actually a matter of a hastle for the end user. That's where I would agree with Hans' comments quite earlier. Putting it in the kernel doesn't make it any more or less of a hassle for the end-user than getting distro support. I remember downloading a different set of Debian floppies which supported XFS, before XFS was mainstream. In that sense, it's somewhat done already -- there is a Gentoo livecd that is kept patched for Reiser4. The problem with Gentoo, of course, is that if you're going to use Gentoo, you're going to be compiling your own kernel. So when it comes down to getting vanilla-sources or gentoo-sources, it wouldn't take much -- just a reiser4-sources, or a separate reiser4-module package. Most people, if they even know what a filesystem or a kernel is, still won't bother compiling their own kernel, you're right. But that means they are more likely to be using a distro-patched kernel than a stock, vanilla one. Well, that's different, and that's the main problem in the linux empowerment that we see around ourselves. It finally revolves around the user, and as harsh as it may seem, it ultimately is the user who decides which fs is better (Give or take, they don't know the difference between a kernel or user-space. or for that matter far more basic things.) If I remember right, SuSe had ReiserFS as the default at one point. If even one moderately popular Linux had Reiser4 as the default FS, it would get a LOT more exposure than it would simply being included (as EXPERIMENTAL, at that) in the vanilla kernel. Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) hate ReiserFS, but there are far more distros than there are popular kernel forks. I'm sure someone will be interested. I do, and that's partly due to the speed of /usr/portage on reiser4, and the easiness of blowing everything and starting from scratch : -) Yes, I love /var/lib/portage/world also. Is /usr/portage still faster on Reiser4? I know it was when I switched, but that was years ago...
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
On Saturday 29 July 2006 23:41, David Masover wrote: > I know Gentoo handles this automatically (emerge nvidia-kernel). I hate to say this again, but its not automatically. It requires more knowledge of ther user, and is more than asking users to patch the kernel. (I am in the support industry and I know of solaris admins, who have used that os for more than ten years ask me what the command patch does and what its impact is to production systems :-) ) >I use an old-style initrd on this box, because my root FS is > on an nvidia RAID, so I have to run a program called "dmraid" before I > mount my root FS -- it's really trivial for me to have Reiser4 as a > module, and I do, despite it being a root FS. As a fact I have it as a root FS (and that too reiser4, and as a module). Surprisingly, this is a wiki article too in http://gentoo-wiki.com, by someone who's more patient enought to write one :-) > I suspect that, all technical, political, and "mine is bigger" arguments > aside, being available as a root FS of a distro, especially a default > FS, would go a long way towards inclusion in the kernel. So all you > have to do is find a reasonably popular and friendly distro, with people > who are (for the moment) easier to deal with than kernel people. Its actually a matter of a hastle for the end user. That's where I would agree with Hans' comments quite earlier. > Most people, if they even know what a filesystem or a kernel is, still > won't bother compiling their own kernel, you're right. But that means > they are more likely to be using a distro-patched kernel than a stock, > vanilla one. Well, that's different, and that's the main problem in the linux empowerment that we see around ourselves. It finally revolves around the user, and as harsh as it may seem, it ultimately is the user who decides which fs is better (Give or take, they don't know the difference between a kernel or user-space. or for that matter far more basic things.) > Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) > hate ReiserFS, but there are far more distros than there are popular > kernel forks. I'm sure someone will be interested. I do, and that's partly due to the speed of /usr/portage on reiser4, and the easiness of blowing everything and starting from scratch : -) Cheers, Sarath -- The gentlemen looked one another over with microscopic carelessness.
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
David Masover writes:
> Nikita Danilov wrote:
>
> > As you see, ext2 code already has multiple file "plugins", with
> > persistent "plugin id" (stored in i_mode field of on-disk struct
> > ext2_inode).
>
> Aha! So here's another question: Is it fair to ask Reiser4 to make its
> plugins generic, or should we be asking ext2/3 first?
ext2/3 plugins are generic: in Linux every file system can implement
per-object behavior by specifying
{file,inode,dentry,address_space}_operations. This mechanism is provided
by VFS (and, in fact, is the only way that VFS interacts with file
system) and is completely generic.
>
Nikita.
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Nikita Danilov wrote: > As you see, ext2 code already has multiple file "plugins", with > persistent "plugin id" (stored in i_mode field of on-disk struct > ext2_inode). Aha! So here's another question: Is it fair to ask Reiser4 to make its plugins generic, or should we be asking ext2/3 first? signature.asc Description: OpenPGP digital signature
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser wrote: > David Masover wrote: > >> If indeed it can be changed easily at all. I think the burden is on >> you to prove that you can change it to be more generic, rather than >> saying "Well, we could do it later, if people want us to..." > > None of the filesystems other than reiser4 have any interest in using > plugins, and this whole argument over how it should be in VFS is > nonsensical because nobody but us has any interest in using the > functionality. The burden is on the generic code authors to prove that > they will ever ever do anything at all besides complain. Frankly, I > don't think they will. I think they will never produce one line of code. I think it's fair to say that 5-10 years from now, with different ext3 maintainers, when the Reiser4 concept has proven itself, people will want plugins for ext3, and the ext3 developers will like the idea. ext* is one of those things that just refuses to die. I use ext3 for my /boot fs, so that I don't have to patch Grub for Reiser4, and so that at least I can mess with the bootloader from any rescue CD if something goes wrong. It's for kind of the same reason that Gentoo builds a 32-bit Grub, even though I'm booting a 64-bit OS -- just in case. I also use ext2 for my initrd. There are other monstrosities that will likely never die, also. ISO9660, UDF, and VFAT probably all have worse storage characteristics than Reiser4, in that as I understand it, they won't pack multiple files into a block. So Reiser4 might even make a good boot cd FS, storing things more efficiently -- but even if I'm right, those three filesystems will last forever, because they are currently well supported on every major OS, and I think one of ISO/UDF is required for DVDs. So for whatever reason someone's using another filesystem, even if all they need is the on-disk format (my reason for ext3 /boot and vfat on USB thumbdrives), I think it's reasonable to expect that they may one day want plugin functionality. People who like Reiser filesystems will do just fine running Reiser4 with a (udf|iso|vfat) storage driver, but people who don't will just want the higher level stuff. You're probably right and this is years of work for something that may not be worth anything, but I think this is what is going through people's heads as they look at this plugin system. So see my comments about distro inclusion. signature.asc Description: OpenPGP digital signature
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Arjan van de Ven wrote: >> Most users not only cannot patch a kernel, they don't know what a patch >> is. It most certainly does. > > > obviously you can provide complete kernels, including precompiled ones. > Most distros have a yum or apt or similar tool to suck down packages, > it's trivial for users to add a site to that, so you could provide > packages if you want and make it easy for them. What's more, many distros patch their kernels extensively. They listen to their users, too. So if there are a lot of users wanting this to be in the kernel, let them complain -- loudly -- to their distro to patch for Reiser4. It could be made even easier than that -- if Reiser4 is really so self-contained, it could be a whole separate set of modules, distributed on its own. Most gamers have to be content with doing something similar with the nvidia drivers -- for different reasons (licensing) but with the same results. I know Gentoo handles this automatically (emerge nvidia-kernel). Hmm, maybe it makes it a pain to have it as a root filesystem, so that really needs distro support. And yet, we have a whole system designed specifically for being able to load modules and tweak settings before the root FS is available. It's called initrd, or more recently, initramfs. I use an old-style initrd on this box, because my root FS is on an nvidia RAID, so I have to run a program called "dmraid" before I mount my root FS -- it's really trivial for me to have Reiser4 as a module, and I do, despite it being a root FS. I suspect that, all technical, political, and "mine is bigger" arguments aside, being available as a root FS of a distro, especially a default FS, would go a long way towards inclusion in the kernel. So all you have to do is find a reasonably popular and friendly distro, with people who are (for the moment) easier to deal with than kernel people. Most people, if they even know what a filesystem or a kernel is, still won't bother compiling their own kernel, you're right. But that means they are more likely to be using a distro-patched kernel than a stock, vanilla one. Is this enough to be "in the jukebox", Hans? Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) hate ReiserFS, but there are far more distros than there are popular kernel forks. I'm sure someone will be interested. That's assuming that making further changes (putting stuff in the VFS) is out of the question (for now). signature.asc Description: OpenPGP digital signature
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser writes: > David Masover wrote: > > > > > If indeed it can be changed easily at all. I think the burden is on > > you to prove that you can change it to be more generic, rather than > > saying "Well, we could do it later, if people want us to..." > > None of the filesystems other than reiser4 have any interest in using > plugins, and this whole argument over how it should be in VFS is > nonsensical because nobody but us has any interest in using the > functionality. The burden is on the generic code authors to prove that > they will ever ever do anything at all besides complain. Frankly, I > don't think they will. I think they will never produce one line of code. > > Please cite one ext3 developer who is signed up to implement ext3 using > plugins if they are supported by VFS. In fact, they all do: struct inode_operations ext2_file_inode_operations; struct inode_operations ext2_dir_inode_operations; struct inode_operations ext2_special_inode_operations; struct inode_operations ext2_symlink_inode_operations; struct inode_operations ext2_fast_symlink_inode_operations; As you see, ext2 code already has multiple file "plugins", with persistent "plugin id" (stored in i_mode field of on-disk struct ext2_inode). > > Hans > Nikita.
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Mike and Lukasz, please post your email to not just reiserfs-list, where only the reiserfs team will read it, but also to lkml if you could, please? Thanks for your support, user opinions count for a lot on lkml.
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Jeff Garzik wrote: > > Using guilt as an argument in a technical discussion is a flashing red > sign that says "I have no technical rebuttal" Wow, that is really nervy. Let's recap this all: * reiser4 has a 2x performance advantage over the next fastest FS (ext3), and when compression ships in a month that will double again as well as save space. See http://www.namesys.com/benchmarks.html, and then ask the reiserfs-list@namesys.com whether those benchmarks are fair representations of their experiences. This is in a field where a 25% advantage is a hard won big deal. * we described our plugin architecture in 2001. No other FS developers were interested, only the users were, and it was presented quite a lot. * So we implemented plugins for ourselves, because no other FS developers would possibly have supported us touching their code. (I do not say that they erred in this.) * No one has actually made a serious case for it being genericizable when you get to the details, it is all just handwaving. I'd be surprised if >10% of it was FS independent, and unsurprised if making that 10% FS independent made the code ossified and hard to maintain. I do not in anyway claim that those who choose to implement Reiser4 plugins are not deeply affected by Reiser4 design choices. Most of the value of writing Reiser4 plugins comes from being able to reuse Reiser4 code as you choose to in the process, and if Reiser4 is not to your taste as a whole, then nobody should impose our plugins upon you. VFS is a bad enough straight jacket for FS developers, we don't need even more mandated design decisions for the FS developers to come who will be brighter than us. Actually, I would like to see Nate Diller implement a competing VFS layer, I think he would do a very good job of that. * Here we are today, and Reiser4 plugins work. Now some say that because we did it for Reiser4 and not for every other FS, that we should be excluded from the kernel. So we are supposed to re-implement it as generic code, which will involve years of time, and then finally something will be coded and nobody but us will use it, and then they will tell us that because nobody but us wants to use it it cannot go in. If you disagree, find one ext3 developer who wants to rewrite ext3 to use plugins and change its disk format to do it. And you have the nerve to say that this ever was a technical discussion? Our code measurably works the best. If folks want to imitate it, go ahead, but don't blame us for making our code work without first making those other folks's code work. The technical rebuttal you ask for is http://www.namesys.com/benchmarks.html. The only time this argument gets technical is when akpm is involved. He was right about what should technically be done about batch write, which, by the way, was greeted upon completion with an if only reiser4 uses it then it should not go in response. We are being penalized for thinking too differently, and this whole ping-pong between "no we don't want to do it your way" and "you did it your way for only you, redo it for us even though we won't ever use it" and "oh, you redid it for us but none of us want to use it, so no it is an imposition and cannot go in" is the Kafka-esque manifestation of that. If only reiser4 wants to use something, then just let us do it in our little corner without bothering anybody else. (Though any advice from akpm that he has time for giving us is always welcome.) David, we aren't asking to be in the band, we are asking to be in the jukebox. I think enough users want to go 2x as fast that the users would benefit from our being in the jukebox. Hans
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
> Most users not only cannot patch a kernel, they don't know what a patch > is. It most certainly does. obviously you can provide complete kernels, including precompiled ones. Most distros have a yum or apt or similar tool to suck down packages, it's trivial for users to add a site to that, so you could provide packages if you want and make it easy for them. -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
David Masover wrote: > > If indeed it can be changed easily at all. I think the burden is on > you to prove that you can change it to be more generic, rather than > saying "Well, we could do it later, if people want us to..." None of the filesystems other than reiser4 have any interest in using plugins, and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. Please cite one ext3 developer who is signed up to implement ext3 using plugins if they are supported by VFS. > >> . It also prevents users from getting >> advances they could be getting today, for no reason. > > > It prevents users from doing nothing. Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. Hans
