Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
"Albert D. Cahalan" wrote: > Hans Reiser writes: > > > Tell us what to code for, and so long as it doesn't involve looking > > up files by their 32 bit inode numbers we'll probably be happy to > > code to it. The Neil Brown stuff is already coded for though. > > Next time around, when you update the on-disk format, how about > allowing for such a thing? > > You could have a tree that maps from inode number to whatever > you need to find a file. This shouldn't affect much more than > file creation and deletion. Maybe it will allow for a more > robust fsck as well, helping to justify the cost. > > It would be really nice to be able to find all filenames that > refer to a given inode number. It would have a significant performance impact and use disk space. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
On Friday, May 11, 2001 04:00:20 AM -0700 Hans Reiser <[EMAIL PROTECTED]> wrote: > Alan Cox wrote: > >> > Are you referring to Neil Brown's nfs operations patch as being as >> > ugly as hell, or something else? Just want to understand what you are >> > saying before arguing. >> >> Andi has sent me some stuff to look at. He listed four implementations >> and I've only seen two of them > > did you see an implementation which adds operations to VFS and is written > by Neil Brown (with reiserfs portions by Chris and Nikita)? I coded up a mixture of Andi's 2.2.x apis and Neil's 2.4.x stuff and sent it out for review a little while ago. It isn't as good as Neil's stuff, but it doesn't require changing the other filesystems. If it looked good to the NFS guys and the other FS guys don't hate it, I'll push it around for testing/inclusion. This would be my preferred solution right now, since it could also work for AFS. -chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
On Friday, May 11, 2001 04:00:20 AM -0700 Hans Reiser [EMAIL PROTECTED] wrote: Alan Cox wrote: Are you referring to Neil Brown's nfs operations patch as being as ugly as hell, or something else? Just want to understand what you are saying before arguing. Andi has sent me some stuff to look at. He listed four implementations and I've only seen two of them did you see an implementation which adds operations to VFS and is written by Neil Brown (with reiserfs portions by Chris and Nikita)? I coded up a mixture of Andi's 2.2.x apis and Neil's 2.4.x stuff and sent it out for review a little while ago. It isn't as good as Neil's stuff, but it doesn't require changing the other filesystems. If it looked good to the NFS guys and the other FS guys don't hate it, I'll push it around for testing/inclusion. This would be my preferred solution right now, since it could also work for AFS. -chris - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
Albert D. Cahalan wrote: Hans Reiser writes: Tell us what to code for, and so long as it doesn't involve looking up files by their 32 bit inode numbers we'll probably be happy to code to it. The Neil Brown stuff is already coded for though. Next time around, when you update the on-disk format, how about allowing for such a thing? You could have a tree that maps from inode number to whatever you need to find a file. This shouldn't affect much more than file creation and deletion. Maybe it will allow for a more robust fsck as well, helping to justify the cost. It would be really nice to be able to find all filenames that refer to a given inode number. It would have a significant performance impact and use disk space. Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
On Friday, May 11, 2001 12:07:08 PM -0700 Hans Reiser <[EMAIL PROTECTED]> wrote: > "Albert D. Cahalan" wrote: > >> Hans Reiser writes: >> >> > Tell us what to code for, and so long as it doesn't involve looking >> > up files by their 32 bit inode numbers we'll probably be happy to >> > code to it. The Neil Brown stuff is already coded for though. >> >> Next time around, when you update the on-disk format, how about >> allowing for such a thing? >> >> You could have a tree that maps from inode number to whatever >> you need to find a file. This shouldn't affect much more than >> file creation and deletion. Maybe it will allow for a more >> robust fsck as well, helping to justify the cost. >> >> It would be really nice to be able to find all filenames that >> refer to a given inode number. > > It would have a significant performance impact and use disk space. inode numbers have in the past been enough to find the object in the filesystem, and more than a few applications have counted on this. In my mind, the difference between an interesting research project and a real professional grade product is compatibility with these kinds of traditional expectations. I think reiserfs has been lucky that we've managed to work around the inode number problem so far (with help from Neil, Andi and others), but the number of hours wasted on cramming the 64bit key into various applications has been huge. It only makes a speed difference because the original format relies on it. When redoing the format for v4, this kind of thing should at least be considered. -chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
> Does NFS server support for one of the included FSes not belong in the > kernel? or is it that a better way is possible and this ugly one should not > be included? If the latter, has anyone told Hans how to do it 'right' so he > can assign someone to the task? The patch Andi forwarded me for NFS still isnt too great but it meets the requirements in that it doesnt touch non reiserfs file systems and its fairly easy to verify that the NFS code paths are not changed for other file systems - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, May 11, 2001 at 05:04:10PM +0100, Alan Cox wrote: > > I think with the growing acceptance of ReiserFS in the Linux > > community, it is tiresome to have to apply a patch again and again > > just to get working NFS. 2.2 NFS horrors all over again. > > The zero copy patches were basically self contained and tested for 6 months. > The reiserfs NFS hacks are ugly as hell and dont belong in the base kernel. > There is a difference. Does NFS server support for one of the included FSes not belong in the kernel? or is it that a better way is possible and this ugly one should not be included? If the latter, has anyone told Hans how to do it 'right' so he can assign someone to the task? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
Hans Reiser writes: > Tell us what to code for, and so long as it doesn't involve looking > up files by their 32 bit inode numbers we'll probably be happy to > code to it. The Neil Brown stuff is already coded for though. Next time around, when you update the on-disk format, how about allowing for such a thing? You could have a tree that maps from inode number to whatever you need to find a file. This shouldn't affect much more than file creation and deletion. Maybe it will allow for a more robust fsck as well, helping to justify the cost. It would be really nice to be able to find all filenames that refer to a given inode number. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
Alan Cox wrote: > > Are you referring to Neil Brown's nfs operations patch as being as ugly as > > hell, or something else? Just want to understand what you are saying before > > arguing. > > Andi has sent me some stuff to look at. He listed four implementations and I've > only seen two of them did you see an implementation which adds operations to VFS and is written by Neil Brown (with reiserfs portions by Chris and Nikita)? > > > > NFS is ugly as hell, and we just try to conform to whatever is the latest trend > > expected to be accepted since I really don't care so long as it works and > > doesn't uglify ReiserFS more than necessary. If you have another approach, one > > that is less ugly, please let us know. This is the first I have heard someone > > Oh believe me we agree in great detail where the -problem- is. Unfortunately > the spec is kind of stuck. Its finding a minimally invasive solution for 2.4 > pending fixing it properly in 2.5 > > Alan Tell us what to code for, and so long as it doesn't involve looking up files by their 32 bit inode numbers we'll probably be happy to code to it. The Neil Brown stuff is already coded for though. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
> Are you referring to Neil Brown's nfs operations patch as being as ugly as > hell, or something else? Just want to understand what you are saying before > arguing. Andi has sent me some stuff to look at. He listed four implementations and I've only seen two of them > NFS is ugly as hell, and we just try to conform to whatever is the latest trend > expected to be accepted since I really don't care so long as it works and > doesn't uglify ReiserFS more than necessary. If you have another approach, one > that is less ugly, please let us know. This is the first I have heard someone Oh believe me we agree in great detail where the -problem- is. Unfortunately the spec is kind of stuck. Its finding a minimally invasive solution for 2.4 pending fixing it properly in 2.5 Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Alan Cox wrote: > > I think with the growing acceptance of ReiserFS in the Linux > > community, it is tiresome to have to apply a patch again and again > > just to get working NFS. 2.2 NFS horrors all over again. > > The zero copy patches were basically self contained and tested for 6 months. > The reiserfs NFS hacks are ugly as hell and dont belong in the base kernel. > There is a difference. > > Alan > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ Are you referring to Neil Brown's nfs operations patch as being as ugly as hell, or something else? Just want to understand what you are saying before arguing. NFS is ugly as hell, and we just try to conform to whatever is the latest trend expected to be accepted since I really don't care so long as it works and doesn't uglify ReiserFS more than necessary. If you have another approach, one that is less ugly, please let us know. This is the first I have heard someone complain, I thought his patch was liked by Linus architecturally and that it would be going in sometime real soon now (which is why we coded for it). Can you articulate why you dislike it in more detail? Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, 11 May 2001, Tony Hoyle wrote: > ls can't access the files either, so I don't see how that could rectify > anything. The entire directory becomes inaccessible. This happened to > /lib once. Nasty. No-one can access the files once the caches are hosed. Purge the inode/dentry caches and retry. > I'd like to be able to use something like reiserfs, especially when > developing (it reduces boot time a lot). However to call it 'stable' on > 2.4.4 is simply wrong. If/when the nfs fix gets merged and tested > *then* it stands a chance of being called stable. Does that actually apply to 2.4.4 or rather to 2.2.19? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: > It's probably not. vs-13048 can usually be rectified (ugly, slow but > usually works on machines even with 256 MB RAM and 1/2 GB swap) by ls > -laR / or treescan -stat /. ls can't access the files either, so I don't see how that could rectify anything. The entire directory becomes inaccessible. This happened to /lib once. Nasty. I'd like to be able to use something like reiserfs, especially when developing (it reduces boot time a lot). However to call it 'stable' on 2.4.4 is simply wrong. If/when the nfs fix gets merged and tested *then* it stands a chance of being called stable. Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
"Henning P. Schmiedehausen" wrote: > Chris Mason <[EMAIL PROTECTED]> writes: > > >It requires explicit changes to each filesystem that wants to work over > >NFS, and is a somewhat large change. > > Come on, we got zerocopy TCP pushed into a stable kernel release with > the words "get over it". > > So please, push this patch to Linus; I really like to "get over it". > > I think with the growing acceptance of ReiserFS in the Linux > community, it is tiresome to have to apply a patch again and again > just to get working NFS. 2.2 NFS horrors all over again. > > Regards > Henning > -- > Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer > INTERMETA - Gesellschaft fuer Mehrwertdienste mbH [EMAIL PROTECTED] > > Am Schwabachgrund 22 Fon.: 09131 / 50654-0 [EMAIL PROTECTED] > D-91054 Buckenhof Fax.: 09131 / 50654-20 > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ nikita, ask Neil what the timeline is for it going into Linux. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
> I think with the growing acceptance of ReiserFS in the Linux > community, it is tiresome to have to apply a patch again and again > just to get working NFS. 2.2 NFS horrors all over again. The zero copy patches were basically self contained and tested for 6 months. The reiserfs NFS hacks are ugly as hell and dont belong in the base kernel. There is a difference. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, 11 May 2001, Tony Hoyle wrote: > Matthias Andree wrote: > > > You're not getting data loss, but access denied, when hitting > > incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less > > of a problem. Please search the reiserfs list archives for details. > > vs-13048 is a good search term, I believe. > > Data is lost: That's not true, at least not in my case. Data is temporarily inaccessible, and in extreme cases, the Kernel might panic, but I never suffered from permanent damage. There might be other reasons, old kernel, old patch, old user-space utilities, hardware flaws. I'm not a good friend of ReiserFS on Linux 2.2 in production, but let's please keep honest, regardless of how big the anger may have grown. > Root can't access the files. True, catches EACCES. > Reiserfsck can't repair the files. Also true, because they're not broken. > The files that are corrupted are unrelated to the ones exported over NFS > (which makes me wonder if it's the same bug). It's probably not. vs-13048 can usually be rectified (ugly, slow but usually works on machines even with 256 MB RAM and 1/2 GB swap) by ls -laR / or treescan -stat /. > File corruption would begin a couple of hours after the volume was > formatted, and become catastrophic within a couple of days. > > Until the fix is merged I'm not going within 100 miles of reiserfs! Well, it would be really a good idea to consider if the patch might be merged into ReiserFS 3.5.33, but AFAICS, the ReiserFS team have focused on getting 3.6 solid. Hans, does ReiserFS 3.6 suffer from vs-13048 or similar problems with NFS in plain Linux 2.4.4 or does ist still need patches as 3.5.x does? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Tony Hoyle wrote: > Matthias Andree wrote: > > > You're not getting data loss, but access denied, when hitting > > incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less > > of a problem. Please search the reiserfs list archives for details. > > vs-13048 is a good search term, I believe. > > Data is lost: > > Root can't access the files. > Reiserfsck can't repair the files. > The files that are corrupted are unrelated to the ones exported over NFS > (which makes me wonder if it's the same bug). > > File corruption would begin a couple of hours after the volume was > formatted, and become catastrophic within a couple of days. > > Until the fix is merged I'm not going within 100 miles of reiserfs! > > Tony > > -- > Where a calculator on the ENIAC is equpped with 18,000 vaccuum > tubes and weighs 30 tons, computers in the future may have only > 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. > -- Popular Mechanics, March 1949 > > [EMAIL PROTECTED] > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ there is a patch, it is on our website, what is the problem? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: > You're not getting data loss, but access denied, when hitting > incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less > of a problem. Please search the reiserfs list archives for details. > vs-13048 is a good search term, I believe. Data is lost: Root can't access the files. Reiserfsck can't repair the files. The files that are corrupted are unrelated to the ones exported over NFS (which makes me wonder if it's the same bug). File corruption would begin a couple of hours after the volume was formatted, and become catastrophic within a couple of days. Until the fix is merged I'm not going within 100 miles of reiserfs! Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Friday, May 11, 2001 01:39:13 AM +0200 Matthias Andree <[EMAIL PROTECTED]> wrote: > On Thu, 10 May 2001, Hans Reiser wrote: > >> > Hmm... Reiserfs is incompatible with knfsd? That might explain the >> >> we have a patch on our website. > > I'm always wondering why the patch hasn't been merged. Is it so > dangerous to apply in that it might distract other pieces of the system? > Has anyone reported increased problems after applying the patch? It requires explicit changes to each filesystem that wants to work over NFS, and is a somewhat large change. -chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On linux-kernel, [EMAIL PROTECTED] (Andi Kleen) wrote: [...] : If /arc is not on a different hd it is probably a good idea to make : sure test.tar.gz is small enough to fit into memory and has been read : at least once to be cache hot (that was the case with my test tar). : Otherwise you're testing how fast the hd can seek between the two places : and how far XFS and ext2 are away, and both are not very interesting. hda: IBM-DTLA-307030, ATA DISK drive hdc: ST34312A, ATA DISK drive hda: 60036480 sectors (30739 MB) w/1916KiB Cache, CHS=3737/255/63, UDMA(66) hdc: 8420832 sectors (4311 MB) w/512KiB Cache, CHS=524/255/63, UDMA(33) /arc is logical volume on hda, /mobile is partition on hdc (mobile - becouse it's on disk in mobile rack ;)), so test is good. Soon I will test this on SCSI disk. -- Daniel Podlejski <[EMAIL PROTECTED]> ... A blind man kneels on broken class Building the bars of his own case ... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, May 11, 2001 at 12:23:09AM +0200, Daniel Podlejski wrote: > > ext2: > > root@witch:/mobile:# time tar xzf /arc/test.tar.gz If /arc is not on a different hd it is probably a good idea to make sure test.tar.gz is small enough to fit into memory and has been read at least once to be cache hot (that was the case with my test tar). Otherwise you're testing how fast the hd can seek between the two places and how far XFS and ext2 are away, and both are not very interesting. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, May 11, 2001 at 12:23:09AM +0200, Daniel Podlejski wrote: ext2: root@witch:/mobile:# time tar xzf /arc/test.tar.gz If /arc is not on a different hd it is probably a good idea to make sure test.tar.gz is small enough to fit into memory and has been read at least once to be cache hot (that was the case with my test tar). Otherwise you're testing how fast the hd can seek between the two places and how far XFS and ext2 are away, and both are not very interesting. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On linux-kernel, [EMAIL PROTECTED] (Andi Kleen) wrote: [...] : If /arc is not on a different hd it is probably a good idea to make : sure test.tar.gz is small enough to fit into memory and has been read : at least once to be cache hot (that was the case with my test tar). : Otherwise you're testing how fast the hd can seek between the two places : and how far XFS and ext2 are away, and both are not very interesting. hda: IBM-DTLA-307030, ATA DISK drive hdc: ST34312A, ATA DISK drive hda: 60036480 sectors (30739 MB) w/1916KiB Cache, CHS=3737/255/63, UDMA(66) hdc: 8420832 sectors (4311 MB) w/512KiB Cache, CHS=524/255/63, UDMA(33) /arc is logical volume on hda, /mobile is partition on hdc (mobile - becouse it's on disk in mobile rack ;)), so test is good. Soon I will test this on SCSI disk. -- Daniel Podlejski [EMAIL PROTECTED] ... A blind man kneels on broken class Building the bars of his own case ... - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Friday, May 11, 2001 01:39:13 AM +0200 Matthias Andree [EMAIL PROTECTED] wrote: On Thu, 10 May 2001, Hans Reiser wrote: Hmm... Reiserfs is incompatible with knfsd? That might explain the we have a patch on our website. I'm always wondering why the patch hasn't been merged. Is it so dangerous to apply in that it might distract other pieces of the system? Has anyone reported increased problems after applying the patch? It requires explicit changes to each filesystem that wants to work over NFS, and is a somewhat large change. -chris - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: You're not getting data loss, but access denied, when hitting incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less of a problem. Please search the reiserfs list archives for details. vs-13048 is a good search term, I believe. Data is lost: Root can't access the files. Reiserfsck can't repair the files. The files that are corrupted are unrelated to the ones exported over NFS (which makes me wonder if it's the same bug). File corruption would begin a couple of hours after the volume was formatted, and become catastrophic within a couple of days. Until the fix is merged I'm not going within 100 miles of reiserfs! Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Tony Hoyle wrote: Matthias Andree wrote: You're not getting data loss, but access denied, when hitting incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less of a problem. Please search the reiserfs list archives for details. vs-13048 is a good search term, I believe. Data is lost: Root can't access the files. Reiserfsck can't repair the files. The files that are corrupted are unrelated to the ones exported over NFS (which makes me wonder if it's the same bug). File corruption would begin a couple of hours after the volume was formatted, and become catastrophic within a couple of days. Until the fix is merged I'm not going within 100 miles of reiserfs! Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ there is a patch, it is on our website, what is the problem? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, 11 May 2001, Tony Hoyle wrote: Matthias Andree wrote: You're not getting data loss, but access denied, when hitting incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less of a problem. Please search the reiserfs list archives for details. vs-13048 is a good search term, I believe. Data is lost: That's not true, at least not in my case. Data is temporarily inaccessible, and in extreme cases, the Kernel might panic, but I never suffered from permanent damage. There might be other reasons, old kernel, old patch, old user-space utilities, hardware flaws. I'm not a good friend of ReiserFS on Linux 2.2 in production, but let's please keep honest, regardless of how big the anger may have grown. Root can't access the files. True, catches EACCES. Reiserfsck can't repair the files. Also true, because they're not broken. The files that are corrupted are unrelated to the ones exported over NFS (which makes me wonder if it's the same bug). It's probably not. vs-13048 can usually be rectified (ugly, slow but usually works on machines even with 256 MB RAM and 1/2 GB swap) by ls -laR / or treescan -stat /. File corruption would begin a couple of hours after the volume was formatted, and become catastrophic within a couple of days. Until the fix is merged I'm not going within 100 miles of reiserfs! Well, it would be really a good idea to consider if the patch might be merged into ReiserFS 3.5.33, but AFAICS, the ReiserFS team have focused on getting 3.6 solid. Hans, does ReiserFS 3.6 suffer from vs-13048 or similar problems with NFS in plain Linux 2.4.4 or does ist still need patches as 3.5.x does? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
I think with the growing acceptance of ReiserFS in the Linux community, it is tiresome to have to apply a patch again and again just to get working NFS. 2.2 NFS horrors all over again. The zero copy patches were basically self contained and tested for 6 months. The reiserfs NFS hacks are ugly as hell and dont belong in the base kernel. There is a difference. Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, 11 May 2001, Tony Hoyle wrote: ls can't access the files either, so I don't see how that could rectify anything. The entire directory becomes inaccessible. This happened to /lib once. Nasty. No-one can access the files once the caches are hosed. Purge the inode/dentry caches and retry. I'd like to be able to use something like reiserfs, especially when developing (it reduces boot time a lot). However to call it 'stable' on 2.4.4 is simply wrong. If/when the nfs fix gets merged and tested *then* it stands a chance of being called stable. Does that actually apply to 2.4.4 or rather to 2.2.19? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Alan Cox wrote: I think with the growing acceptance of ReiserFS in the Linux community, it is tiresome to have to apply a patch again and again just to get working NFS. 2.2 NFS horrors all over again. The zero copy patches were basically self contained and tested for 6 months. The reiserfs NFS hacks are ugly as hell and dont belong in the base kernel. There is a difference. Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ Are you referring to Neil Brown's nfs operations patch as being as ugly as hell, or something else? Just want to understand what you are saying before arguing. NFS is ugly as hell, and we just try to conform to whatever is the latest trend expected to be accepted since I really don't care so long as it works and doesn't uglify ReiserFS more than necessary. If you have another approach, one that is less ugly, please let us know. This is the first I have heard someone complain, I thought his patch was liked by Linus architecturally and that it would be going in sometime real soon now (which is why we coded for it). Can you articulate why you dislike it in more detail? Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Are you referring to Neil Brown's nfs operations patch as being as ugly as hell, or something else? Just want to understand what you are saying before arguing. Andi has sent me some stuff to look at. He listed four implementations and I've only seen two of them NFS is ugly as hell, and we just try to conform to whatever is the latest trend expected to be accepted since I really don't care so long as it works and doesn't uglify ReiserFS more than necessary. If you have another approach, one that is less ugly, please let us know. This is the first I have heard someone Oh believe me we agree in great detail where the -problem- is. Unfortunately the spec is kind of stuck. Its finding a minimally invasive solution for 2.4 pending fixing it properly in 2.5 Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
Alan Cox wrote: Are you referring to Neil Brown's nfs operations patch as being as ugly as hell, or something else? Just want to understand what you are saying before arguing. Andi has sent me some stuff to look at. He listed four implementations and I've only seen two of them did you see an implementation which adds operations to VFS and is written by Neil Brown (with reiserfs portions by Chris and Nikita)? NFS is ugly as hell, and we just try to conform to whatever is the latest trend expected to be accepted since I really don't care so long as it works and doesn't uglify ReiserFS more than necessary. If you have another approach, one that is less ugly, please let us know. This is the first I have heard someone Oh believe me we agree in great detail where the -problem- is. Unfortunately the spec is kind of stuck. Its finding a minimally invasive solution for 2.4 pending fixing it properly in 2.5 Alan Tell us what to code for, and so long as it doesn't involve looking up files by their 32 bit inode numbers we'll probably be happy to code to it. The Neil Brown stuff is already coded for though. Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
Hans Reiser writes: Tell us what to code for, and so long as it doesn't involve looking up files by their 32 bit inode numbers we'll probably be happy to code to it. The Neil Brown stuff is already coded for though. Next time around, when you update the on-disk format, how about allowing for such a thing? You could have a tree that maps from inode number to whatever you need to find a file. This shouldn't affect much more than file creation and deletion. Maybe it will allow for a more robust fsck as well, helping to justify the cost. It would be really nice to be able to find all filenames that refer to a given inode number. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Fri, May 11, 2001 at 05:04:10PM +0100, Alan Cox wrote: I think with the growing acceptance of ReiserFS in the Linux community, it is tiresome to have to apply a patch again and again just to get working NFS. 2.2 NFS horrors all over again. The zero copy patches were basically self contained and tested for 6 months. The reiserfs NFS hacks are ugly as hell and dont belong in the base kernel. There is a difference. Does NFS server support for one of the included FSes not belong in the kernel? or is it that a better way is possible and this ugly one should not be included? If the latter, has anyone told Hans how to do it 'right' so he can assign someone to the task? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Does NFS server support for one of the included FSes not belong in the kernel? or is it that a better way is possible and this ugly one should not be included? If the latter, has anyone told Hans how to do it 'right' so he can assign someone to the task? The patch Andi forwarded me for NFS still isnt too great but it meets the requirements in that it doesnt touch non reiserfs file systems and its fairly easy to verify that the NFS code paths are not changed for other file systems - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [reiserfs-dev] Re: reiserfs, xfs, ext2, ext3
On Friday, May 11, 2001 12:07:08 PM -0700 Hans Reiser [EMAIL PROTECTED] wrote: Albert D. Cahalan wrote: Hans Reiser writes: Tell us what to code for, and so long as it doesn't involve looking up files by their 32 bit inode numbers we'll probably be happy to code to it. The Neil Brown stuff is already coded for though. Next time around, when you update the on-disk format, how about allowing for such a thing? You could have a tree that maps from inode number to whatever you need to find a file. This shouldn't affect much more than file creation and deletion. Maybe it will allow for a more robust fsck as well, helping to justify the cost. It would be really nice to be able to find all filenames that refer to a given inode number. It would have a significant performance impact and use disk space. inode numbers have in the past been enough to find the object in the filesystem, and more than a few applications have counted on this. In my mind, the difference between an interesting research project and a real professional grade product is compatibility with these kinds of traditional expectations. I think reiserfs has been lucky that we've managed to work around the inode number problem so far (with help from Neil, Andi and others), but the number of hours wasted on cramming the 64bit key into various applications has been huge. It only makes a speed difference because the original format relies on it. When redoing the format for v4, this kind of thing should at least be considered. /soapbox -chris - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, 10 May 2001, Hans Reiser wrote: > > Hmm... Reiserfs is incompatible with knfsd? That might explain the > > we have a patch on our website. I'm always wondering why the patch hasn't been merged. Is it so dangerous to apply in that it might distract other pieces of the system? Has anyone reported increased problems after applying the patch? -- Matthias Andree - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, 10 May 2001, Tony Hoyle wrote: > Hmm... Reiserfs is incompatible with knfsd? That might explain the > massive data loss I was getting with reiserfs (basically I'd have to > reformat and reinstall every couple of weeks). The machine this was > happening with also exports my apt cache for the rest of the network. You're not getting data loss, but access denied, when hitting incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less of a problem. Please search the reiserfs list archives for details. vs-13048 is a good search term, I believe. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On linux-kernel, [EMAIL PROTECTED] (Andi Kleen) wrote: : On one not very scientific test: unpacking and deleting a cache hot 40MB/230MB : gzipped/unzipped tar on ext2 and xfs on a IDE drive on a lowend SMP box. : : XFS (very recent 2.4.4 CVS, filesystem created with mkxfs defaults) : : > time tar xzf ~ak/src.tgz : real1m58.125s : user0m16.410s : sys 0m44.350s : > time rm -rf src/ : real0m50.344s : user0m0.190s : sys 0m13.950s : : ext2 (on same kernel as above) : : > time tar xzf ~ak/src.tgz : : real1m26.126s : user0m16.100s : sys 0m36.080s : : > time rm -rf src/ : : real0m1.085s : user0m0.160s : sys 0m0.930s And another test: ext2: root@witch:/mobile:# time tar xzf /arc/test.tar.gz real5m25.249s user1m33.430s sys 0m31.710s root@witch:/mobile:# time rm -rf test/ real0m8.843s user0m0.000s sys 0m0.420s xfs: root@witch:/mobile:# time tar xzf /arc/test.tar.gz real5m23.744s user1m34.800s sys 0m39.100s root@witch:/mobile:# time rm -rf test/ real0m1.364s user0m0.030s sys 0m0.430s test.tar.gz is tree created by script: #!/bin/bash for lev1 in aa bb cc dd ee ff gg hh ii jj kk ll do mkdir $lev1 cd $lev1 for lev2 in 0 1 2 3 4 5 6 7 do mkdir $lev2 cd $lev2 for fname in a b c d e f g h i do dd if=/dev/urandom of=$fname bs=4k count=1024 done cd .. done cd .. done -- Daniel Podlejski <[EMAIL PROTECTED]> ... When you're talkin to yourself and nobody's home You can fool yourself, you came in this world alone ... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, May 10, 2001 at 01:44:53PM +0200, Matthias Andree wrote: [snip] > If you're deploying a cache partition such as /var/squid (possibly > having log files in another /var/log partition on another disk drive), > what's the point about not running (e. g.) mke2fs and squid -z on boot, > as well as mounting the system partitions (/usr) read-only (prevents > fsck on next reboot)? mke2fs is faster than reiserfs recovery probably > ;-) A while ago I configured a few squid boxes which ran off of a read-only system. Mke2fs is actually unacceptably slow on large file systems, faster then fsck, but still time consuming. I found that zeroing out the disk, then formating it and saving the non-zero blocks, replaying them on reboot to be an acceptable solution. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Mié 09 May 2001 21:32, Joel Jaeggli wrote: > I have a proxy server that's been running 2.4.3pre4 with reiserfs for the > partitions on the cache disks. it has an uptime of 43 days at this point. > it wasn't very stable at all (two crashes in one week) with 2.4.2. I'll be > building 2.4.4 something when I get back from ghana to the US, but I don't > want to reboot it onto a fresh kernel while I'm 11,000 miles away, serial > console notwithstanding. > > Overall I'm of the belief that reiserfs is robust enough for mainstream > use, and it's significantly faster than ext2 for the squid box, you do as > usal need to be a bit selective about what kernel you choose to run. Could you give un information on the hardware you're using for that proxy? Saludos... :-) -- El mejor sistema operativo es aquel que te da de comer. Cuida tu dieta. - Martin Marques |[EMAIL PROTECTED] Programador, Administrador | Centro de Telematica Universidad Nacional del Litoral - - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thursday 10 May 2001 12:19, Pekka Pietikainen wrote: > Here's some very unscientific numbers I've measured (ancient 4G SCSI > drive on a dual pII/450), XFS 1.0-pre2 and reiserfs is > the version in that kernel. The first bit is from tiobench, the > multiple files one is a simple 30-line program that creates tons of > 1k files and then removes them. > > XFS > > Create 2 files: 116.882449 > Unlink 2 files: 47.449201 > > reiserfs > > Create 2 files: 17.862143 > Unlink 2 files: 9.487520 > > ext2 > > Create 2 files: 248.758925 > Unlink 2 files: 2.287174 Whoops! The create test is exactly the case my index patch accelerates, trying it is highly recommended: http://nl.linux.org/~phillips/htree/dx.testme-2.4.3-3 To apply: cd source/tree zcat ext2-dir-patch-S4.gz | patch -p1 cat dx.pcache-2.4.4 | patch -p0 Or for the all-in-page-cache version (needs Al Viro's patch, see the READ.ME at http://nl.linux.org/~phillips/htree/): http://nl.linux.org/~phillips/htree/dx.pcache-2.4.4-4 The testme version is easier to apply but the pcache version is more like what the final form will be (waiting for Al's patch to get into Linus's tree so I can de-fork this). -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Tony Hoyle wrote: > Matthias Andree wrote: > > > ext3fs has never given me any problems, but I did not have it in > > production use where I discovered major ReiserFS <-> kNFSd > > incompatibilities. ext3 has a 0.0.x version number which suggests it's > > not meant for production use. > > Hmm... Reiserfs is incompatible with knfsd? That might explain the > massive data loss I was getting with reiserfs (basically I'd have to > reformat and reinstall every couple of weeks). The machine this was > happening with also exports my apt cache for the rest of the network. > > Tony > > -- > Where a calculator on the ENIAC is equpped with 18,000 vaccuum > tubes and weighs 30 tons, computers in the future may have only > 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. > -- Popular Mechanics, March 1949 > > [EMAIL PROTECTED] > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ we have a patch on our website. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: > > If you're deploying a cache partition such as /var/squid (possibly > having log files in another /var/log partition on another disk drive), > what's the point about not running (e. g.) mke2fs and squid -z on boot, > as well as mounting the system partitions (/usr) read-only (prevents > fsck on next reboot)? mke2fs is faster than reiserfs recovery probably > ;-) For that particular application of squid, it happens we are much much faster than ext2, if you apply all the right tunings and especially if you apply the reiserfs_raw patch. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
I would encourage all of you to consider using a fractal fileset generator such as reiserfs_fract_tree.c such as we use for mongo.pl which we use for internal benchmarking. You can get a copy at www.namesys.com in the benchmarking section, and then tune it as suits your needs. I think that one needs randomly generated non-uniform sized files and directories and directory trees to have a good benchmark. We gratefully accept enhancements of it. Now that we are stable we can go back to tuning things like large file performance. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
> quite a bit of scope for improvement. Commercial caching systems have > demonstrated thoughput of thousands of requests/s with similar > hardware, but I suspect Tux-ification of Squid will be necessary to not at all, search for X15 in april/may linux-kernel archives. most of the specific improvements tux made have been reduced to improvements for the general case, hence squid (or equivalent) could probably improve a fair amount. j. -- "Bobby, jiggle Grandpa's rat so it looks alive, please" -- gary larson - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On 05/10, Matthias Andree rearranged the electrons to read: > On Wed, 09 May 2001, Mart?n Marqu?s wrote: > > > There has also been lots of talks about reiserfs being the cause of > > some data lose and performance lose (not sure about this last one). > > I never experienced ReiserFS data loss, but I did experience read > performance loss over ext2fs and switched that file system back to ext2. > The ReiserFS people could not reproduce the problem, so I'm not sure > what was the actual cause. > > ext3fs has never given me any problems, but I did not have it in > production use where I discovered major ReiserFS <-> kNFSd > incompatibilities. ext3 has a 0.0.x version number which suggests it's > not meant for production use. > > XFS is claimed to work with NFS, but not currently availabe for Linux > 2.4.4. $ uname -r 2.4.4-ac5.xfs $ uptime 9:20am up 1 day, 18:28, 4 users, load average: 0.00, 0.00, 0.00 -- Hob Goblin [EMAIL PROTECTED] A society that will trade a little liberty for a little order will lose both and deserve neither. - Thomas Jefferson - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: > ext3fs has never given me any problems, but I did not have it in > production use where I discovered major ReiserFS <-> kNFSd > incompatibilities. ext3 has a 0.0.x version number which suggests it's > not meant for production use. Hmm... Reiserfs is incompatible with knfsd? That might explain the massive data loss I was getting with reiserfs (basically I'd have to reformat and reinstall every couple of weeks). The machine this was happening with also exports my apt cache for the rest of the network. Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Content-Type: text/plain; charset=us-ascii Just to pick up on this thread, to operate the JANET Web Cache Service (proxy caching thingy used by a lot of ac.uk folk) we currently run Squid on top of 2.2.18 + Linux Virtual Server 1.0.5 + ReiserFS 3.5.31 and the Magic-SysRq-over-serial-console patch. In almost a year of operation (admittedly covering a couple of earlier kernel/ReiserFS/... combos, and gradually migrated to our 40-odd cache servers) we've only ever had a couple of wibbles which directly implicated ReiserFS - and these on servers which later turned out to have hardware problems. Lockups on the older 3c905 and eepro100 drivers have caused us some problems historically, though. As the JANET backbone has migrated to 2.5Gbit/s we've seen our daily transaction rate rise to over 100m URLs, with one peak of 120m. Moving our cache filesystems from ext2 to ReiserFS almost tripled the number of requests we could process per second before a cache box became overloaded, and our fastest machines (6 x 10,000 RPM Ultra160 SCSI cache disks) are now shipping some 230 requests/s with around 50ms latency for a cache hit. FWIW, we're currently using Squid 2.2.STABLE5-hno.2202 and using ReiserFS as a regular filesystem rather than raw, so there ought to be quite a bit of scope for improvement. Commercial caching systems have demonstrated thoughput of thousands of requests/s with similar hardware, but I suspect Tux-ification of Squid will be necessary to catch up with them. This (in the form of our old chum the proprietary loadable kernel module :-) seems to be the direction the commercial vendors are going in to get maximal performance out of their Linux ports... Cheers, Martin -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.4 (GNU/Linux) Comment: Exmh version 2.1.1 10/15/1999 + martin iD8DBQE6+pQyVw+hz3xBJfQRApYsAKCuy1yPe/KarPluSeTB6OKgmfQ+8wCfYPi2 li9nQT05bhlj7Us3fXf3+l8= =biJi -END PGP SIGNATURE- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wed, May 09, 2001 at 04:25:40PM -0500, Steve Lord wrote: > > > > > XFS is very fast most of the time (deleting a file is so slow its like us > > ing > > old BSD systems). Im not familiar enough with its behaviour under Linux yet. > > Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 > seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu > linux box with a not very exciting scsi drive. On one not very scientific test: unpacking and deleting a cache hot 40MB/230MB gzipped/unzipped tar on ext2 and xfs on a IDE drive on a lowend SMP box. XFS (very recent 2.4.4 CVS, filesystem created with mkxfs defaults) > time tar xzf ~ak/src.tgz real1m58.125s user0m16.410s sys 0m44.350s > time rm -rf src/ real0m50.344s user0m0.190s sys 0m13.950s ext2 (on same kernel as above) > time tar xzf ~ak/src.tgz real1m26.126s user0m16.100s sys 0m36.080s > time rm -rf src/ real0m1.085s user0m0.160s sys 0m0.930s ext2 seems to be faster and the difference on deletion is dramatic, so at least here it looks like Alan's statement is true. The test did not involve very large files, the biggest files in the tar are a few hundred K with most of them being much smaller. The values stay similar over multiple runs. I did not do any comparisons recently with reiserfs, but at least in the past reiserfs usually came out ahead of ext2 for similar tests (especially being much faster for deletion) -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
john slee wrote: > > On Wed, May 09, 2001 at 10:38:14AM +0300, Mart?n Marqu?s wrote: > > We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to > > come so we can change our proxy server, that will run on Linux with Squid. > > One disk will go inside (I think?) and the other 4 on a tower conected to the > > RAID, which will be have the cache of the squid server. > > that's a pretty huge cache, have you considered using 8*9gb disks > instead of 4*18? what sort of request throughput/latency are you > aiming for? as the number of requests grows, disk seek times are > a very real problem. > > > One of my partners thinks that we should use reiserfs on all the server (the > > partitions of the Linux distro, and the cache partitions), and I found out > > that reiserfs has had lots of bugs, and is marked as experimental in kernel > > well, lots of bugs in reiserfs have been fixed.. obviously there are > more bugs to come (as always), but on the whole a lot of people are very > happy with it. there certainly haven't been many posts of reiserfs > corruption lately at all on linux-kernel. > > > 2.4.4. Not to mention that the people of RH discourage there users from using > > it. > > they do? > > > So what I want is to know which is the status of this 3 journaling FS. Which > > is the one we should look for? > > xfs, while wonderful, probably isn't what you're looking for. AFAIK it > is intended more for very very very large files. > > > I think that the data lose is not significant in a proxy cache, if the FS is > > really fast, as is said reiserfs is. > > data loss is always significant. consider the case where you are forced > to rebuild the filesystem squid's cache directories reside on... > admittedly it is an extreme case, but it is a possibility all the same. > If you worry about that, put the squid cache in a fs of its own. If it ever dies - use mkfs and restart with an empty cache. No need to spend a long time fsck'ing something with a limited lifetime that can be re-fetched. Helge Hafting - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wed, May 09, 2001 at 04:25:40PM -0500, Steve Lord wrote: > > > > > XFS is very fast most of the time (deleting a file is so slow its like us > > ing > > old BSD systems). Im not familiar enough with its behaviour under Linux yet. > > Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 > seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu > linux box with a not very exciting scsi drive. Here's some very unscientific numbers I've measured (ancient 4G SCSI drive on a dual pII/450), XFS 1.0-pre2 and reiserfs is the version in that kernel. The first bit is from tiobench, the multiple files one is a simple 30-line program that creates tons of 1k files and then removes them. XFS File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- . 25640961 9.601 8.36% 0.644 1.48% 9.367 20.8% 0.587 1.12% . 25640962 7.201 6.51% 0.672 1.50% 9.323 24.3% 0.582 1.63% . 25640964 6.867 6.25% 0.693 1.46% 9.280 27.0% 0.590 1.91% . 25640968 6.669 6.29% 0.708 1.57% 9.215 31.4% 0.597 1.99% Create 2 files: 116.882449 Unlink 2 files: 47.449201 reiserfs File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- . 25640961 9.591 11.9% 0.480 1.53% 4.506 21.4% 0.581 1.67% . 25640962 7.176 8.91% 0.557 1.56% 5.559 30.3% 0.579 1.96% . 25640964 6.509 8.34% 0.593 1.69% 6.142 36.9% 0.580 1.96% . 25640968 5.602 7.17% 0.621 1.84% 6.430 40.5% 0.581 1.99% Create 2 files: 17.862143 Unlink 2 files: 9.487520 ext2 File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- . 25640961 9.533 7.26% 0.472 1.11% 4.505 7.77% 0.582 1.11% . 25640962 7.274 5.56% 0.559 1.16% 5.667 12.2% 0.584 1.14% . 25640964 6.377 4.84% 0.600 1.13% 6.209 15.2% 0.587 1.40% . 25640968 5.613 4.33% 0.623 1.16% 6.433 17.4% 0.589 1.58% Create 2 files: 248.758925 Unlink 2 files: 2.287174 -- Pekka Pietikainen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, May 10, 2001 at 01:44:53PM +0200, Matthias Andree wrote: [snip] If you're deploying a cache partition such as /var/squid (possibly having log files in another /var/log partition on another disk drive), what's the point about not running (e. g.) mke2fs and squid -z on boot, as well as mounting the system partitions (/usr) read-only (prevents fsck on next reboot)? mke2fs is faster than reiserfs recovery probably ;-) A while ago I configured a few squid boxes which ran off of a read-only system. Mke2fs is actually unacceptably slow on large file systems, faster then fsck, but still time consuming. I found that zeroing out the disk, then formating it and saving the non-zero blocks, replaying them on reboot to be an acceptable solution. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On linux-kernel, [EMAIL PROTECTED] (Andi Kleen) wrote: : On one not very scientific test: unpacking and deleting a cache hot 40MB/230MB : gzipped/unzipped tar on ext2 and xfs on a IDE drive on a lowend SMP box. : : XFS (very recent 2.4.4 CVS, filesystem created with mkxfs defaults) : : time tar xzf ~ak/src.tgz : real1m58.125s : user0m16.410s : sys 0m44.350s : time rm -rf src/ : real0m50.344s : user0m0.190s : sys 0m13.950s : : ext2 (on same kernel as above) : : time tar xzf ~ak/src.tgz : : real1m26.126s : user0m16.100s : sys 0m36.080s : : time rm -rf src/ : : real0m1.085s : user0m0.160s : sys 0m0.930s And another test: ext2: root@witch:/mobile:# time tar xzf /arc/test.tar.gz real5m25.249s user1m33.430s sys 0m31.710s root@witch:/mobile:# time rm -rf test/ real0m8.843s user0m0.000s sys 0m0.420s xfs: root@witch:/mobile:# time tar xzf /arc/test.tar.gz real5m23.744s user1m34.800s sys 0m39.100s root@witch:/mobile:# time rm -rf test/ real0m1.364s user0m0.030s sys 0m0.430s test.tar.gz is tree created by script: #!/bin/bash for lev1 in aa bb cc dd ee ff gg hh ii jj kk ll do mkdir $lev1 cd $lev1 for lev2 in 0 1 2 3 4 5 6 7 do mkdir $lev2 cd $lev2 for fname in a b c d e f g h i do dd if=/dev/urandom of=$fname bs=4k count=1024 done cd .. done cd .. done -- Daniel Podlejski [EMAIL PROTECTED] ... When you're talkin to yourself and nobody's home You can fool yourself, you came in this world alone ... - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, 10 May 2001, Tony Hoyle wrote: Hmm... Reiserfs is incompatible with knfsd? That might explain the massive data loss I was getting with reiserfs (basically I'd have to reformat and reinstall every couple of weeks). The machine this was happening with also exports my apt cache for the rest of the network. You're not getting data loss, but access denied, when hitting incompatibilities, and it looks like it hits 2.2 hard while 2.4 is less of a problem. Please search the reiserfs list archives for details. vs-13048 is a good search term, I believe. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, 10 May 2001, Hans Reiser wrote: Hmm... Reiserfs is incompatible with knfsd? That might explain the we have a patch on our website. I'm always wondering why the patch hasn't been merged. Is it so dangerous to apply in that it might distract other pieces of the system? Has anyone reported increased problems after applying the patch? -- Matthias Andree - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
quite a bit of scope for improvement. Commercial caching systems have demonstrated thoughput of thousands of requests/s with similar hardware, but I suspect Tux-ification of Squid will be necessary to not at all, search for X15 in april/may linux-kernel archives. most of the specific improvements tux made have been reduced to improvements for the general case, hence squid (or equivalent) could probably improve a fair amount. j. -- Bobby, jiggle Grandpa's rat so it looks alive, please -- gary larson - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Mié 09 May 2001 21:32, Joel Jaeggli wrote: I have a proxy server that's been running 2.4.3pre4 with reiserfs for the partitions on the cache disks. it has an uptime of 43 days at this point. it wasn't very stable at all (two crashes in one week) with 2.4.2. I'll be building 2.4.4 something when I get back from ghana to the US, but I don't want to reboot it onto a fresh kernel while I'm 11,000 miles away, serial console notwithstanding. Overall I'm of the belief that reiserfs is robust enough for mainstream use, and it's significantly faster than ext2 for the squid box, you do as usal need to be a bit selective about what kernel you choose to run. Could you give un information on the hardware you're using for that proxy? Saludos... :-) -- El mejor sistema operativo es aquel que te da de comer. Cuida tu dieta. - Martin Marques |[EMAIL PROTECTED] Programador, Administrador | Centro de Telematica Universidad Nacional del Litoral - - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wed, May 09, 2001 at 04:25:40PM -0500, Steve Lord wrote: XFS is very fast most of the time (deleting a file is so slow its like us ing old BSD systems). Im not familiar enough with its behaviour under Linux yet. Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu linux box with a not very exciting scsi drive. On one not very scientific test: unpacking and deleting a cache hot 40MB/230MB gzipped/unzipped tar on ext2 and xfs on a IDE drive on a lowend SMP box. XFS (very recent 2.4.4 CVS, filesystem created with mkxfs defaults) time tar xzf ~ak/src.tgz real1m58.125s user0m16.410s sys 0m44.350s time rm -rf src/ real0m50.344s user0m0.190s sys 0m13.950s ext2 (on same kernel as above) time tar xzf ~ak/src.tgz real1m26.126s user0m16.100s sys 0m36.080s time rm -rf src/ real0m1.085s user0m0.160s sys 0m0.930s ext2 seems to be faster and the difference on deletion is dramatic, so at least here it looks like Alan's statement is true. The test did not involve very large files, the biggest files in the tar are a few hundred K with most of them being much smaller. The values stay similar over multiple runs. I did not do any comparisons recently with reiserfs, but at least in the past reiserfs usually came out ahead of ext2 for similar tests (especially being much faster for deletion) -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wed, May 09, 2001 at 04:25:40PM -0500, Steve Lord wrote: XFS is very fast most of the time (deleting a file is so slow its like us ing old BSD systems). Im not familiar enough with its behaviour under Linux yet. Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu linux box with a not very exciting scsi drive. Here's some very unscientific numbers I've measured (ancient 4G SCSI drive on a dual pII/450), XFS 1.0-pre2 and reiserfs is the version in that kernel. The first bit is from tiobench, the multiple files one is a simple 30-line program that creates tons of 1k files and then removes them. XFS File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- . 25640961 9.601 8.36% 0.644 1.48% 9.367 20.8% 0.587 1.12% . 25640962 7.201 6.51% 0.672 1.50% 9.323 24.3% 0.582 1.63% . 25640964 6.867 6.25% 0.693 1.46% 9.280 27.0% 0.590 1.91% . 25640968 6.669 6.29% 0.708 1.57% 9.215 31.4% 0.597 1.99% Create 2 files: 116.882449 Unlink 2 files: 47.449201 reiserfs File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- . 25640961 9.591 11.9% 0.480 1.53% 4.506 21.4% 0.581 1.67% . 25640962 7.176 8.91% 0.557 1.56% 5.559 30.3% 0.579 1.96% . 25640964 6.509 8.34% 0.593 1.69% 6.142 36.9% 0.580 1.96% . 25640968 5.602 7.17% 0.621 1.84% 6.430 40.5% 0.581 1.99% Create 2 files: 17.862143 Unlink 2 files: 9.487520 ext2 File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- . 25640961 9.533 7.26% 0.472 1.11% 4.505 7.77% 0.582 1.11% . 25640962 7.274 5.56% 0.559 1.16% 5.667 12.2% 0.584 1.14% . 25640964 6.377 4.84% 0.600 1.13% 6.209 15.2% 0.587 1.40% . 25640968 5.613 4.33% 0.623 1.16% 6.433 17.4% 0.589 1.58% Create 2 files: 248.758925 Unlink 2 files: 2.287174 -- Pekka Pietikainen - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
john slee wrote: On Wed, May 09, 2001 at 10:38:14AM +0300, Mart?n Marqu?s wrote: We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to come so we can change our proxy server, that will run on Linux with Squid. One disk will go inside (I think?) and the other 4 on a tower conected to the RAID, which will be have the cache of the squid server. that's a pretty huge cache, have you considered using 8*9gb disks instead of 4*18? what sort of request throughput/latency are you aiming for? as the number of requests grows, disk seek times are a very real problem. One of my partners thinks that we should use reiserfs on all the server (the partitions of the Linux distro, and the cache partitions), and I found out that reiserfs has had lots of bugs, and is marked as experimental in kernel well, lots of bugs in reiserfs have been fixed.. obviously there are more bugs to come (as always), but on the whole a lot of people are very happy with it. there certainly haven't been many posts of reiserfs corruption lately at all on linux-kernel. 2.4.4. Not to mention that the people of RH discourage there users from using it. they do? So what I want is to know which is the status of this 3 journaling FS. Which is the one we should look for? xfs, while wonderful, probably isn't what you're looking for. AFAIK it is intended more for very very very large files. I think that the data lose is not significant in a proxy cache, if the FS is really fast, as is said reiserfs is. data loss is always significant. consider the case where you are forced to rebuild the filesystem squid's cache directories reside on... admittedly it is an extreme case, but it is a possibility all the same. If you worry about that, put the squid cache in a fs of its own. If it ever dies - use mkfs and restart with an empty cache. No need to spend a long time fsck'ing something with a limited lifetime that can be re-fetched. Helge Hafting - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Content-Type: text/plain; charset=us-ascii Just to pick up on this thread, to operate the JANET Web Cache Service (proxy caching thingy used by a lot of ac.uk folk) we currently run Squid on top of 2.2.18 + Linux Virtual Server 1.0.5 + ReiserFS 3.5.31 and the Magic-SysRq-over-serial-console patch. In almost a year of operation (admittedly covering a couple of earlier kernel/ReiserFS/... combos, and gradually migrated to our 40-odd cache servers) we've only ever had a couple of wibbles which directly implicated ReiserFS - and these on servers which later turned out to have hardware problems. Lockups on the older 3c905 and eepro100 drivers have caused us some problems historically, though. As the JANET backbone has migrated to 2.5Gbit/s we've seen our daily transaction rate rise to over 100m URLs, with one peak of 120m. Moving our cache filesystems from ext2 to ReiserFS almost tripled the number of requests we could process per second before a cache box became overloaded, and our fastest machines (6 x 10,000 RPM Ultra160 SCSI cache disks) are now shipping some 230 requests/s with around 50ms latency for a cache hit. FWIW, we're currently using Squid 2.2.STABLE5-hno.2202 and using ReiserFS as a regular filesystem rather than raw, so there ought to be quite a bit of scope for improvement. Commercial caching systems have demonstrated thoughput of thousands of requests/s with similar hardware, but I suspect Tux-ification of Squid will be necessary to catch up with them. This (in the form of our old chum the proprietary loadable kernel module :-) seems to be the direction the commercial vendors are going in to get maximal performance out of their Linux ports... Cheers, Martin -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.4 (GNU/Linux) Comment: Exmh version 2.1.1 10/15/1999 + martin iD8DBQE6+pQyVw+hz3xBJfQRApYsAKCuy1yPe/KarPluSeTB6OKgmfQ+8wCfYPi2 li9nQT05bhlj7Us3fXf3+l8= =biJi -END PGP SIGNATURE- - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: ext3fs has never given me any problems, but I did not have it in production use where I discovered major ReiserFS - kNFSd incompatibilities. ext3 has a 0.0.x version number which suggests it's not meant for production use. Hmm... Reiserfs is incompatible with knfsd? That might explain the massive data loss I was getting with reiserfs (basically I'd have to reformat and reinstall every couple of weeks). The machine this was happening with also exports my apt cache for the rest of the network. Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On 05/10, Matthias Andree rearranged the electrons to read: On Wed, 09 May 2001, Mart?n Marqu?s wrote: There has also been lots of talks about reiserfs being the cause of some data lose and performance lose (not sure about this last one). I never experienced ReiserFS data loss, but I did experience read performance loss over ext2fs and switched that file system back to ext2. The ReiserFS people could not reproduce the problem, so I'm not sure what was the actual cause. ext3fs has never given me any problems, but I did not have it in production use where I discovered major ReiserFS - kNFSd incompatibilities. ext3 has a 0.0.x version number which suggests it's not meant for production use. XFS is claimed to work with NFS, but not currently availabe for Linux 2.4.4. $ uname -r 2.4.4-ac5.xfs $ uptime 9:20am up 1 day, 18:28, 4 users, load average: 0.00, 0.00, 0.00 -- Hob Goblin [EMAIL PROTECTED] A society that will trade a little liberty for a little order will lose both and deserve neither. - Thomas Jefferson - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
I would encourage all of you to consider using a fractal fileset generator such as reiserfs_fract_tree.c such as we use for mongo.pl which we use for internal benchmarking. You can get a copy at www.namesys.com in the benchmarking section, and then tune it as suits your needs. I think that one needs randomly generated non-uniform sized files and directories and directory trees to have a good benchmark. We gratefully accept enhancements of it. Now that we are stable we can go back to tuning things like large file performance. Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Matthias Andree wrote: If you're deploying a cache partition such as /var/squid (possibly having log files in another /var/log partition on another disk drive), what's the point about not running (e. g.) mke2fs and squid -z on boot, as well as mounting the system partitions (/usr) read-only (prevents fsck on next reboot)? mke2fs is faster than reiserfs recovery probably ;-) For that particular application of squid, it happens we are much much faster than ext2, if you apply all the right tunings and especially if you apply the reiserfs_raw patch. Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Tony Hoyle wrote: Matthias Andree wrote: ext3fs has never given me any problems, but I did not have it in production use where I discovered major ReiserFS - kNFSd incompatibilities. ext3 has a 0.0.x version number which suggests it's not meant for production use. Hmm... Reiserfs is incompatible with knfsd? That might explain the massive data loss I was getting with reiserfs (basically I'd have to reformat and reinstall every couple of weeks). The machine this was happening with also exports my apt cache for the rest of the network. Tony -- Where a calculator on the ENIAC is equpped with 18,000 vaccuum tubes and weighs 30 tons, computers in the future may have only 1,000 vaccuum tubes and perhaps weigh 1 1\2 tons. -- Popular Mechanics, March 1949 [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ we have a patch on our website. Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thursday 10 May 2001 12:19, Pekka Pietikainen wrote: Here's some very unscientific numbers I've measured (ancient 4G SCSI drive on a dual pII/450), XFS 1.0-pre2 and reiserfs is the version in that kernel. The first bit is from tiobench, the multiple files one is a simple 30-line program that creates tons of 1k files and then removes them. XFS Create 2 files: 116.882449 Unlink 2 files: 47.449201 reiserfs Create 2 files: 17.862143 Unlink 2 files: 9.487520 ext2 Create 2 files: 248.758925 Unlink 2 files: 2.287174 Whoops! The create test is exactly the case my index patch accelerates, trying it is highly recommended: http://nl.linux.org/~phillips/htree/dx.testme-2.4.3-3 To apply: cd source/tree zcat ext2-dir-patch-S4.gz | patch -p1 cat dx.pcache-2.4.4 | patch -p0 Or for the all-in-page-cache version (needs Al Viro's patch, see the READ.ME at http://nl.linux.org/~phillips/htree/): http://nl.linux.org/~phillips/htree/dx.pcache-2.4.4-4 The testme version is easier to apply but the pcache version is more like what the final form will be (waiting for Al's patch to get into Linus's tree so I can de-fork this). -- Daniel - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Daniel writes: > The ext2 indexing patch is apparently stable but it's still pre-alpha > until the hash function is finalized. I could see using it to run > performance tests of ext2+indexing against the alternatives, but only > if you are prepared to rerun mke2fs later. Then there is the matter of > making fsck index-aware. As it stands now, if fsck finds an index it > will remove it. Actually, I would think e2fsck will simply refuse to run if the COMPAT flag is set... > It would be great to see a table of ReiserFS/XFS/Ext2+index performance > results. Well, to make it really fair it should be Ext3+index so I'd > better add 'backport the patch to 2.2' or 'bug Stephen and friends to > hurry up' to my to-do list. Actually, Andrew Morton has a nearly working ext3 implementation for 2.4. It is at the stage where it works most of the time, but we are not yet sure that everything that needs to be journaled is. Cheers, Andreas -- Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto, \ would they cancel out, leaving him still hungry?" http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3 (simple benchmarks)
> It would be great to see a table of ReiserFS/XFS/Ext2+index performance > results. Well, to make it really fair it should be Ext3+index so I'd > better add 'backport the patch to 2.2' or 'bug Stephen and friends to > hurry up' to my to-do list. You can find a simple benchmark (an average of three samples) among reiser, ext2, xfs and fat32 under Linux: http://bulma.lug.net/body.phtml?nIdNoticia=626 Although is Spanish, the tables are easy to understand. The benchmark was carried up by Guillem Cantallops, student of the University of Balearics Islands and member or the local LUG... BASIC WORDS ;-) Escritura: Writing Lectura:Reading Borrado:Deletion Copia: Copy Extracción: Extraction Regards, --ricardo http://m3d.uib.es/~gallir/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wed, May 09, 2001 at 04:19:53PM -0700, Dan Hollis wrote: > On Thu, 10 May 2001, Daniel Phillips wrote: > > It would be great to see a table of ReiserFS/XFS/Ext2+index performance > > results. Well, to make it really fair it should be Ext3+index so I'd > > better add 'backport the patch to 2.2' or 'bug Stephen and friends to > > hurry up' to my to-do list. > > Is the IBM JFS at a testable state yet, or is it still 'bleedin beta'? It is not stable enough to use in production yet. It is certainly stable enough to test/benchmark. They released a new patch today which is supposed to fix a long standing deadlock, so thats good news. Give it a shot. Jim - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, 10 May 2001, Daniel Phillips wrote: > It would be great to see a table of ReiserFS/XFS/Ext2+index performance > results. Well, to make it really fair it should be Ext3+index so I'd > better add 'backport the patch to 2.2' or 'bug Stephen and friends to > hurry up' to my to-do list. Is the IBM JFS at a testable state yet, or is it still 'bleedin beta'? -Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wednesday 09 May 2001 16:49, john slee wrote: > On Wed, May 09, 2001 at 10:38:14AM +0300, Mart?n Marqu?s wrote: > > We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb > > scsi disks to come so we can change our proxy server, that will run > > on Linux with Squid. One disk will go inside (I think?) and the > > other 4 on a tower conected to the RAID, which will be have the > > cache of the squid server. [...] > also appropriate could be ext2 with daniel phillips' directory > indexing patches. The ext2 indexing patch is apparently stable but it's still pre-alpha until the hash function is finalized. I could see using it to run performance tests of ext2+indexing against the alternatives, but only if you are prepared to rerun mke2fs later. Then there is the matter of making fsck index-aware. As it stands now, if fsck finds an index it will remove it. But by all means please test the patch: http://nl.linux.org/~phillips/htree/dx.testme-2.4.4-4 It would be great to see a table of ReiserFS/XFS/Ext2+index performance results. Well, to make it really fair it should be Ext3+index so I'd better add 'backport the patch to 2.2' or 'bug Stephen and friends to hurry up' to my to-do list. -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Hans Reiser wrote: > XFS used to have the performance problems that Alan described but fixed them > in > the linux port, yes? > > Hans Hmm, we do things somewhat differently on linux, but I suspect most of it is due to hardware getting faster underneath us. Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Steve Lord wrote: > > > > XFS is very fast most of the time (deleting a file is so slow its like us > > ing > > old BSD systems). Im not familiar enough with its behaviour under Linux yet. > > Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 > seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu > linux box with a not very exciting scsi drive. > > > > > What you might want to do is to make a partition for 'mystery journalling fs' > > and benchmark a bit. > > > > Alan > > > > I agree with Alan here, the only sure fire way to find out which filesystem > will work best for your application is to try it out. I have found reiserfs > to be very fast in some tests, especially those operating on lots of small > files, but contrary to some peoples, belief XFS is good for a lot more than > just messing with Gbyte long data files. > > Steve Lord > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ XFS used to have the performance problems that Alan described but fixed them in the linux port, yes? Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
> > XFS is very fast most of the time (deleting a file is so slow its like us > ing > old BSD systems). Im not familiar enough with its behaviour under Linux yet. Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu linux box with a not very exciting scsi drive. > > What you might want to do is to make a partition for 'mystery journalling fs' > and benchmark a bit. > > Alan > I agree with Alan here, the only sure fire way to find out which filesystem will work best for your application is to try it out. I have found reiserfs to be very fast in some tests, especially those operating on lots of small files, but contrary to some peoples, belief XFS is good for a lot more than just messing with Gbyte long data files. Steve Lord - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Daniel Podlejski wrote: > In linux-kernel, [EMAIL PROTECTED] wrote: > : We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to > : come so we can change our proxy server, that will run on Linux with Squid. > : One disk will go inside (I think?) and the other 4 on a tower conected to the > : RAID, which will be have the cache of the squid server. > > Reiser is not good idea for squid cache. Why ? Squids cache is hashed > by default to minimize "big directory effect". Reiserfs is fast on > realy big directories, but it eat more cpu than ext2. On high loaded > cache servers cpu utilization on reiser can be realy big. > Better way is make more subdirectories in squid cache, and use > filesystem, that eat less of cpu. > > -- > Daniel Podlejski <[EMAIL PROTECTED]> >... I am immortal, I have inside me blood of kings, >I have no rival, No man can bemy equal ... > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ Nice theory, but the benchmarks say the opposite. Turn off tails, use reiserfs raw, and we are the fastest way to do squid, which is still way slower than the proprietary solutions that don't use squid. (squid's user space architecture is poorly threaded, and that becomes the bottleneck.) Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
In linux-kernel, [EMAIL PROTECTED] wrote: : We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to : come so we can change our proxy server, that will run on Linux with Squid. : One disk will go inside (I think?) and the other 4 on a tower conected to the : RAID, which will be have the cache of the squid server. Reiser is not good idea for squid cache. Why ? Squids cache is hashed by default to minimize "big directory effect". Reiserfs is fast on realy big directories, but it eat more cpu than ext2. On high loaded cache servers cpu utilization on reiser can be realy big. Better way is make more subdirectories in squid cache, and use filesystem, that eat less of cpu. -- Daniel Podlejski <[EMAIL PROTECTED]> ... I am immortal, I have inside me blood of kings, I have no rival, No man can bemy equal ... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
I have a proxy server that's been running 2.4.3pre4 with reiserfs for the partitions on the cache disks. it has an uptime of 43 days at this point. it wasn't very stable at all (two crashes in one week) with 2.4.2. I'll be building 2.4.4 something when I get back from ghana to the US, but I don't want to reboot it onto a fresh kernel while I'm 11,000 miles away, serial console notwithstanding. Overall I'm of the belief that reiserfs is robust enough for mainstream use, and it's significantly faster than ext2 for the squid box, you do as usal need to be a bit selective about what kernel you choose to run. On Wed, 9 May 2001, Martín Marqués wrote: > Hi, > > We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to > come so we can change our proxy server, that will run on Linux with Squid. > One disk will go inside (I think?) and the other 4 on a tower conected to the > RAID, which will be have the cache of the squid server. > > One of my partners thinks that we should use reiserfs on all the server (the > partitions of the Linux distro, and the cache partitions), and I found out > that reiserfs has had lots of bugs, and is marked as experimental in kernel > 2.4.4. Not to mention that the people of RH discourage there users from using > it. > > There has also been lots of talks about reiserfs being the cause of some data > lose and performance lose (not sure about this last one). > > So what I want is to know which is the status of this 3 journaling FS. Which > is the one we should look for? > > I think that the data lose is not significant in a proxy cache, if the FS is > really fast, as is said reiserfs is. > > Saludos... :-) > > -- -- Joel Jaeggli [EMAIL PROTECTED] Academic User Services [EMAIL PROTECTED] PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- It is clear that the arm of criticism cannot replace the criticism of arms. Karl Marx -- Introduction to the critique of Hegel's Philosophy of the right, 1843. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Alan Cox wrote: > > that reiserfs has had lots of bugs, and is marked as experimental in kernel > > 2.4.4. Not to mention that the people of RH discourage there users from using > > it. > > At the time Red Hat 7.1 was mastered Reiserfs was not stable. The reiserfs in > the RH kernel has some of the tail fixes but newer ones are not present. Also > it had other problems then: the fsck tool was useless, it didnt work on > big endian machines (eg PPC, S/390). > > If Hans sent me a patch removing the experimental tag from Reiserfs the only > thing that would make me hesitate the slightest from applying it would be the > endianness thing, and thats not enough to stop it being applied. Jeff Mahoney has a patch in progress for this, he currently has the kernel code working, but needs to do the utilities. I would hesitate to put the endianness fixes in before 2.5.1 just because I am conservative about disturbing stable code. There exists one known bug which one user has hit which required a major code change to fix. We are now testing the code, and are in the ironic situation of hesitating to merge in a bug fix out of fear that the bugfix code is large and untested, and it might have bugs that more than one user will hit.:-/ I think we are going to make the new code an option until it has been extensively tested. I think that 2.4.4 is stable, and I say this based upon us getting lots of users with hardware bugs and none with bugs not fixed in 2.4.4 in the entire time since 2.4.4 was released. > > > > There has also been lots of talks about reiserfs being the cause of some data > > lose and performance lose (not sure about this last one). > > If you are running 2.4.4/2.4.4-ac/2.4.5pre I believe all the relevant reiserfs > patches are applied. The new fsck seems to work a lot better too. The limiters > right now are: > - You need a patch for NFS (its on their site no big deal) > - You can only use little endian boxes (x86 for you so ok) you also need a patch for quotas. > > > I think that the data lose is not significant in a proxy cache, if the FS is > > really fast, as is said reiserfs is. you can ask nikita for a copy of reiserfs_raw, a version of reiserfs designed for squid. It is substantially faster. Hans - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Steve Lord wrote: XFS is very fast most of the time (deleting a file is so slow its like us ing old BSD systems). Im not familiar enough with its behaviour under Linux yet. Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu linux box with a not very exciting scsi drive. What you might want to do is to make a partition for 'mystery journalling fs' and benchmark a bit. Alan I agree with Alan here, the only sure fire way to find out which filesystem will work best for your application is to try it out. I have found reiserfs to be very fast in some tests, especially those operating on lots of small files, but contrary to some peoples, belief XFS is good for a lot more than just messing with Gbyte long data files. Steve Lord - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ XFS used to have the performance problems that Alan described but fixed them in the linux port, yes? Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Hans Reiser wrote: XFS used to have the performance problems that Alan described but fixed them in the linux port, yes? Hans Hmm, we do things somewhat differently on linux, but I suspect most of it is due to hardware getting faster underneath us. Steve - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
XFS is very fast most of the time (deleting a file is so slow its like us ing old BSD systems). Im not familiar enough with its behaviour under Linux yet. Hmm, I just removed 2.2 Gbytes of data in 3 files in 37 seconds (14.4 seconds system time), not tooo slow. And that is on a pretty vanilla 2 cpu linux box with a not very exciting scsi drive. What you might want to do is to make a partition for 'mystery journalling fs' and benchmark a bit. Alan I agree with Alan here, the only sure fire way to find out which filesystem will work best for your application is to try it out. I have found reiserfs to be very fast in some tests, especially those operating on lots of small files, but contrary to some peoples, belief XFS is good for a lot more than just messing with Gbyte long data files. Steve Lord - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Thu, 10 May 2001, Daniel Phillips wrote: It would be great to see a table of ReiserFS/XFS/Ext2+index performance results. Well, to make it really fair it should be Ext3+index so I'd better add 'backport the patch to 2.2' or 'bug Stephen and friends to hurry up' to my to-do list. Is the IBM JFS at a testable state yet, or is it still 'bleedin beta'? -Dan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Daniel writes: The ext2 indexing patch is apparently stable but it's still pre-alpha until the hash function is finalized. I could see using it to run performance tests of ext2+indexing against the alternatives, but only if you are prepared to rerun mke2fs later. Then there is the matter of making fsck index-aware. As it stands now, if fsck finds an index it will remove it. Actually, I would think e2fsck will simply refuse to run if the COMPAT flag is set... It would be great to see a table of ReiserFS/XFS/Ext2+index performance results. Well, to make it really fair it should be Ext3+index so I'd better add 'backport the patch to 2.2' or 'bug Stephen and friends to hurry up' to my to-do list. Actually, Andrew Morton has a nearly working ext3 implementation for 2.4. It is at the stage where it works most of the time, but we are not yet sure that everything that needs to be journaled is. Cheers, Andreas -- Andreas Dilger \ If a man ate a pound of pasta and a pound of antipasto, \ would they cancel out, leaving him still hungry? http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wednesday 09 May 2001 16:49, john slee wrote: On Wed, May 09, 2001 at 10:38:14AM +0300, Mart?n Marqu?s wrote: We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to come so we can change our proxy server, that will run on Linux with Squid. One disk will go inside (I think?) and the other 4 on a tower conected to the RAID, which will be have the cache of the squid server. [...] also appropriate could be ext2 with daniel phillips' directory indexing patches. The ext2 indexing patch is apparently stable but it's still pre-alpha until the hash function is finalized. I could see using it to run performance tests of ext2+indexing against the alternatives, but only if you are prepared to rerun mke2fs later. Then there is the matter of making fsck index-aware. As it stands now, if fsck finds an index it will remove it. But by all means please test the patch: http://nl.linux.org/~phillips/htree/dx.testme-2.4.4-4 It would be great to see a table of ReiserFS/XFS/Ext2+index performance results. Well, to make it really fair it should be Ext3+index so I'd better add 'backport the patch to 2.2' or 'bug Stephen and friends to hurry up' to my to-do list. -- Daniel - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
In linux-kernel, [EMAIL PROTECTED] wrote: : We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to : come so we can change our proxy server, that will run on Linux with Squid. : One disk will go inside (I think?) and the other 4 on a tower conected to the : RAID, which will be have the cache of the squid server. Reiser is not good idea for squid cache. Why ? Squids cache is hashed by default to minimize big directory effect. Reiserfs is fast on realy big directories, but it eat more cpu than ext2. On high loaded cache servers cpu utilization on reiser can be realy big. Better way is make more subdirectories in squid cache, and use filesystem, that eat less of cpu. -- Daniel Podlejski [EMAIL PROTECTED] ... I am immortal, I have inside me blood of kings, I have no rival, No man can bemy equal ... - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Alan Cox wrote: that reiserfs has had lots of bugs, and is marked as experimental in kernel 2.4.4. Not to mention that the people of RH discourage there users from using it. At the time Red Hat 7.1 was mastered Reiserfs was not stable. The reiserfs in the RH kernel has some of the tail fixes but newer ones are not present. Also it had other problems then: the fsck tool was useless, it didnt work on big endian machines (eg PPC, S/390). If Hans sent me a patch removing the experimental tag from Reiserfs the only thing that would make me hesitate the slightest from applying it would be the endianness thing, and thats not enough to stop it being applied. Jeff Mahoney has a patch in progress for this, he currently has the kernel code working, but needs to do the utilities. I would hesitate to put the endianness fixes in before 2.5.1 just because I am conservative about disturbing stable code. There exists one known bug which one user has hit which required a major code change to fix. We are now testing the code, and are in the ironic situation of hesitating to merge in a bug fix out of fear that the bugfix code is large and untested, and it might have bugs that more than one user will hit.:-/ I think we are going to make the new code an option until it has been extensively tested. I think that 2.4.4 is stable, and I say this based upon us getting lots of users with hardware bugs and none with bugs not fixed in 2.4.4 in the entire time since 2.4.4 was released. There has also been lots of talks about reiserfs being the cause of some data lose and performance lose (not sure about this last one). If you are running 2.4.4/2.4.4-ac/2.4.5pre I believe all the relevant reiserfs patches are applied. The new fsck seems to work a lot better too. The limiters right now are: - You need a patch for NFS (its on their site no big deal) - You can only use little endian boxes (x86 for you so ok) you also need a patch for quotas. I think that the data lose is not significant in a proxy cache, if the FS is really fast, as is said reiserfs is. you can ask nikita for a copy of reiserfs_raw, a version of reiserfs designed for squid. It is substantially faster. Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
I have a proxy server that's been running 2.4.3pre4 with reiserfs for the partitions on the cache disks. it has an uptime of 43 days at this point. it wasn't very stable at all (two crashes in one week) with 2.4.2. I'll be building 2.4.4 something when I get back from ghana to the US, but I don't want to reboot it onto a fresh kernel while I'm 11,000 miles away, serial console notwithstanding. Overall I'm of the belief that reiserfs is robust enough for mainstream use, and it's significantly faster than ext2 for the squid box, you do as usal need to be a bit selective about what kernel you choose to run. On Wed, 9 May 2001, Martín Marqués wrote: Hi, We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to come so we can change our proxy server, that will run on Linux with Squid. One disk will go inside (I think?) and the other 4 on a tower conected to the RAID, which will be have the cache of the squid server. One of my partners thinks that we should use reiserfs on all the server (the partitions of the Linux distro, and the cache partitions), and I found out that reiserfs has had lots of bugs, and is marked as experimental in kernel 2.4.4. Not to mention that the people of RH discourage there users from using it. There has also been lots of talks about reiserfs being the cause of some data lose and performance lose (not sure about this last one). So what I want is to know which is the status of this 3 journaling FS. Which is the one we should look for? I think that the data lose is not significant in a proxy cache, if the FS is really fast, as is said reiserfs is. Saludos... :-) -- -- Joel Jaeggli [EMAIL PROTECTED] Academic User Services [EMAIL PROTECTED] PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- It is clear that the arm of criticism cannot replace the criticism of arms. Karl Marx -- Introduction to the critique of Hegel's Philosophy of the right, 1843. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
Daniel Podlejski wrote: In linux-kernel, [EMAIL PROTECTED] wrote: : We are waiting for a server with dual PIII, RAID 1,0 and 5 18Gb scsi disks to : come so we can change our proxy server, that will run on Linux with Squid. : One disk will go inside (I think?) and the other 4 on a tower conected to the : RAID, which will be have the cache of the squid server. Reiser is not good idea for squid cache. Why ? Squids cache is hashed by default to minimize big directory effect. Reiserfs is fast on realy big directories, but it eat more cpu than ext2. On high loaded cache servers cpu utilization on reiser can be realy big. Better way is make more subdirectories in squid cache, and use filesystem, that eat less of cpu. -- Daniel Podlejski [EMAIL PROTECTED] ... I am immortal, I have inside me blood of kings, I have no rival, No man can bemy equal ... - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ Nice theory, but the benchmarks say the opposite. Turn off tails, use reiserfs raw, and we are the fastest way to do squid, which is still way slower than the proprietary solutions that don't use squid. (squid's user space architecture is poorly threaded, and that becomes the bottleneck.) Hans - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3
On Wed, May 09, 2001 at 04:19:53PM -0700, Dan Hollis wrote: On Thu, 10 May 2001, Daniel Phillips wrote: It would be great to see a table of ReiserFS/XFS/Ext2+index performance results. Well, to make it really fair it should be Ext3+index so I'd better add 'backport the patch to 2.2' or 'bug Stephen and friends to hurry up' to my to-do list. Is the IBM JFS at a testable state yet, or is it still 'bleedin beta'? It is not stable enough to use in production yet. It is certainly stable enough to test/benchmark. They released a new patch today which is supposed to fix a long standing deadlock, so thats good news. Give it a shot. Jim - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: reiserfs, xfs, ext2, ext3 (simple benchmarks)
It would be great to see a table of ReiserFS/XFS/Ext2+index performance results. Well, to make it really fair it should be Ext3+index so I'd better add 'backport the patch to 2.2' or 'bug Stephen and friends to hurry up' to my to-do list. You can find a simple benchmark (an average of three samples) among reiser, ext2, xfs and fat32 under Linux: http://bulma.lug.net/body.phtml?nIdNoticia=626 Although is Spanish, the tables are easy to understand. The benchmark was carried up by Guillem Cantallops, student of the University of Balearics Islands and member or the local LUG... BASIC WORDS ;-) Escritura: Writing Lectura:Reading Borrado:Deletion Copia: Copy Extracción: Extraction Regards, --ricardo http://m3d.uib.es/~gallir/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
