Re: Reiser4. BEST FILESYSTEM EVER.
On 08 Apr 2007 06:32:26 +0200, Christer Weinigel <[EMAIL PROTECTED]> wrote: [EMAIL PROTECTED] writes: > Lennart. Tell me again that these results from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > are not of interest to you. I still don't understand why you have your > head in the sand. Oh, for fucks sake, stop sounding like a broken record. You have repeated the same totally meaningless statistics more times than I care to count. Please shut the fuck up. wow, it's really amazing how reiser4 can still inspire flamewars so easily when Hans isn't even around to antagonize people and escalate things As you discovered yourself (even though you seem to fail to understand the significance of your discovery), bonnie writes files that consist of mostly zeroes. If your normal use cases consist of creating a bunch of files containing zeroes, reiser4 with compression will do great. Just lovely. Except that nobody sane would store a lot of files containing zeroes, except an an excercize in mental masturbation. So the two bonnie benchmarks with lzo and gzip are totally meaningless for any real life usages. yeah, i sure wish Grev was still around running the benchmarks and regression testing, cause I thought she came up with a good, QA oriented mix of real benchmarks. aside from a number of streaming video benchmarks i did, those were the only results i actually trusted to compare reiser4 with other systems. I know Ted doesn't like the Mongo suite, cause it focuses on small files and shows the common weakness of block-aligned storage ... personally i thought it was great for its primary purpose, making sure reiser4 was optimized for its target workload. i also recall that the distribution of small files to large ones in mongo was pulled from some paper out of CMU, but i can't find the reference to that study right now. As for the amount of disk needed to store three kernel trees, the figures you quote show that Reiser4 does tail combining where the tail of multiple files are stored in one disk block. A nice trick that seems save you about 15% disk space compared to ext3. Now you have to realise what that means, it means that if the disk block containing those tails (or any metadata pointing at that block) gets corrupted, instead of just losing one disk block for one file, you will have lost the tail for all the files sharing that disk block. Depending on your personal prioritites, saving 15% of the space may be worth the risk to you, or maybe not. Personally, for the only disk I'm short on space on, I mostly store flac encoded images of my CD collection, and saving 2kByte out of every 300MByte disk simply doesn't make any difference, and I much prefer a stable file system that I can trust not to lose my data. You might make different choices. well, it turns out that reiser4 does things a little differently, since tail packing has bad performance effects (i always turn it off on my reiserfs partitions). Reiser4 guarantees a file will be stored contiguously if it is below a certain size (20K?), and instead stores the whole file unaligned, so that many files can be packed together without slack space. this gives the best of both worlds performance-wise, at the expense of some complicated flush code to pack everything together in the tree before it gets written. that combined with the fine-grained locking scheme (per-node -- reiserfs just has a global lock) is the primary reason the code is so convoluted ... not poor coding. The same goes for just about every feature that you tout, it has its advantages, and it has its disadvantages. Doing compression on data is great if the data you store is compressible, and sucks if it isn't. Doing compression on each disk block and then packing multiple compressed blocks into each physical disk block will probably save some space if the data is compressible, but at the same time it means that you will spend a lot of CPU (and cache footprint) compressing and uncompressing that data. On a single user system where the CPU is mostly idle it might not make much of a difference, on a heavily loaded multiuser system it might do. my understanding of the code is that it uses a heuristic to decide if a file is already compressed, so that the system doesn't waste time on them and simply writes them out directly. there may also be a way to turn it off for certain classes of files, this would be most useful for executables and the like that are frequently mmap()ed and we care more about page-alignment than read bandwidth or data density. edward? Logs can be compressed quite well using a block based compression scheme, but the logs can be compressed even better by doing compression on the whole file with gzip. So what's the best choice, to do transparent compression on the fly giving ok compression or teaching the userspace tools to do compression of old logs and get really good
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
[EMAIL PROTECTED] wrote: On Mon, 09 Apr 2007 00:58:53 +0200, "Richard Knutsson" <[EMAIL PROTECTED]> said: Wow, I'm impressed. Think you got the record on how many mails you referenced to in a reply... TWO actually. I guess you are easily impressed. Oh, took it to be from 5-6 sources... + you have repeated the same statement several times, that is not the best way of convincing people. I know you DON'T believe that, as you are about the tenth person to repeat that "repeating stuff has no effect." Why should we change our response to the same error? The only solution to this loop is when people stops answering you and you "lose". I believe you picked up the "anti-Reiser religion"-phrase from previous rant-wars (otherwise, why does that "religion"-phrase always come up, and (almost) only when dealing with Reiser-fs), and yes, there has been some clashes caused by both sides, so please be careful when dealing with this matter. NO. You people simply come across as zealots who work together, against Reiser4. Hence the term "anti-Reiser religion." Please, don't address someone you meet for the first time as "you people"! Yes, we do _work_ together, it is a community and as a community you have to follow the social rules agreed upon. Without all those pro-Reiser peoples who knew how to work with the rest, there would not be a ResierFS/Reiser3 in the kernel. Unfortunately, Hans is in this case his own worst enemy and has ruffed quite a few feathers over the time. I don't think you would like someone who tells you "if you do it my way, then you are doing it wrong"... But personally, even if I find Hans a bit too strong-headed, he got some interesting design-ideas and the Reiser-filesystem is something I think many find interesting as a concept but not yet trust-worthy for their own machines. Would you be willing to benchmark Reiser4 with some compressed binary-blob and show the time as well as the CPU-usage? I might be. I don't really know how to set it all up. Perhaps if you guided me through it. Am not sure how much help I would be but from the responses to your benchmark-list, there seems to be many who could help you. But first I think you should set up a system to test on, and then after some tests and made the result public, there will (most likely) be people who ask you to test it in some specific way. I may have missed something, but if my room-mate took my harddrive, screwed it open, wrote a love-letter on the disk with a pencil and then returned it (ok, there may be some more plausible reasons for corruption), is the OS really suppose to handle it? Yeah, I can't see how the OS could read the love-letter either. But one thing is for sure. The FS ain't responsible for reading it. And no-one has asked the file-system to _read_ the disk, but to be designed to help restore the file-structure. This I have found to be the main-point people complains about. It is like arguing against air-bags in a car. Of course the car should not be responsible for preventing accidents, but they are designed so _if_ it happens, you should not be totally screwed. Yes, it should not assign any new data to those blocks but should it not also fall into the file-systems domain to be able to restore some/all data? It's a tough ask of any FS. Microsoft's filesystem checker totally roasted all my data on an XP-box last night. Sorry to hear that, but two wrongs does not make it right. Richard Knutsson - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
On Sun, Apr 08, 2007 at 10:14:18PM -0700, [EMAIL PROTECTED] wrote: > NO. You people simply come across as zealots who work together, against > Reiser4. Poor guy ! People are not against Reiser4, they are against the stupid and irritating person who pollutes the lists always sending the same results without any comment because he doesn't even understand the results. Just like the kid on the beach "Look Ma, I found a soft shell!". "Leave it overthere, it's a jellyfish !". "I don't know what a jellyfish is, I will take all those soft shells with me". You keep saying people do not want to read you, but there is nothing to read. In fact, you hope that people will comment on your results so that you will finally understand them, but people keep saying there is nothing to read there. When will you stop annoying people with your noisy toys ? Oh, and please send your conspiracy claims somewhere else. Willy - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
On Mon, 09 Apr 2007 00:58:53 +0200, "Richard Knutsson" <[EMAIL PROTECTED]> said: > Wow, I'm impressed. Think you got the record on how many mails you > referenced to in a reply... TWO actually. I guess you are easily impressed. A simple cut and paste error. > You have got some rude answers and you have called them back on it Yeah, I (fairly closely) mimicked their behavior to make a point. > + you have repeated the same statement several times, that is > not the best way of convincing people. I know you DON'T believe that, as you are about the tenth person to repeat that "repeating stuff has no effect." > I believe you picked up the "anti-Reiser religion"-phrase from previous > rant-wars (otherwise, why does that "religion"-phrase always come up, > and (almost) only when dealing with Reiser-fs), and yes, there has been > some clashes caused by both sides, so please be careful when dealing > with this matter. NO. You people simply come across as zealots who work together, against Reiser4. Hence the term "anti-Reiser religion." > Would you be willing to benchmark Reiser4 with some compressed > binary-blob and show the time as well as the CPU-usage? I might be. I don't really know how to set it all up. Perhaps if you guided me through it. > > > > You deliberately ignored the fact that bad blocks are NOT dealt with by > > the filesystem,... but by the operating system. Like I said: If your > > filesystem is writing to bad blocks, then throw away your operating > > system. > > > I may have missed something, but if my room-mate took my harddrive, > screwed it open, wrote a love-letter on the disk with a pencil and then > returned it (ok, there may be some more plausible reasons for > corruption), is the OS really suppose to handle it? Yeah, I can't see how the OS could read the love-letter either. But one thing is for sure. The FS ain't responsible for reading it. > Yes, it should not > assign any new data to those blocks but should it not also fall into the > file-systems domain to be able to restore some/all data? It's a tough ask of any FS. Microsoft's filesystem checker totally roasted all my data on an XP-box last night. I had used ntfsresize to reduce the partition size and had a power outage. Later, Windows booted, ran the filesystem checker, seemed OK. Next time I boot, all I get is Input/Output error. > > Just my 2c to the pond > Richard Knutsson > Addin my 2c John. -- [EMAIL PROTECTED] -- http://www.fastmail.fm - A no graphics, no pop-ups email service - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
David H. Lynch Jr wrote: Jeff Garzik wrote: David H. Lynch Jr wrote: I'm arguing against circular logic: the claim that one cannot determine reiser4's true usefulness unless its in the tree. The better method is to get a distro to add reiser4, _then_ if it proves worthy add it to the kernel tree. Not the other way around. And is that how other filesystems made it into the tree ? In the case of most major filesystems, yes. Distros are a proving ground for new stuff, not the upstream kernel. I regularly see drivers with very little in the way of testing go straight nearly straight into the tree - without even getting tagged as experimental. Hardware drivers are vastly different from filesystem drivers. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Jeff Garzik wrote: > David H. Lynch Jr wrote: > > I'm arguing against circular logic: the claim that one cannot > determine reiser4's true usefulness unless its in the tree. > > The better method is to get a distro to add reiser4, _then_ if it > proves worthy add it to the kernel tree. > > Not the other way around. And is that how other filesystems made it into the tree ? I am actually trying to be rational about this. But if my logic is circular then the above is even more so. I am aware of distributions that would incorporate Reiser4 if it was in the tree, but not really aware of much that works the other way around. Besides after eliminating say the top 4 distributions, how is it that getting included as a burried alternate in an obscure distribution is going to change the total number of existing Reiser4 users to a meaningful extent. It would be my guess that Reiser4 already has more users than any filesystem has ever had prior to inclusion in the kernel. Aside from a few instances where distributors are also contributors, and a few instances where specific drivers such as gaming video drivers are in extremely high demand, how frequently do drivers go into distributions before they go into the tree ? Is that supposed to be the standard for all drivers ? I regularly see drivers with very little in the way of testing go straight nearly straight into the tree - without even getting tagged as experimental. And what is CONFIG_EXERIMENTAL supposed to be for ? All I am asking is that now that Hans is not pissing all over people that the Reiser4 Filesystem get exactly the same treatment that pretty much every other filesystem already in or slated for inclusion in the kernel has received. I am sure that their are others who may have a more accurate perception of history. But my recollection is that the only filesystem driver that went through a distribution prior to getting into the Kernel was ReiserFS. I am not even asking that it get accepted exactly asis - though it is my perception that despite Hans's pissing and moaning in the end he made most every change that was asked. Regardless, now that you don't have Hans to deal with anymore. Give Namesys an honest and reasonable set of requirements to get a decent review - the same shot any other filesystem would get. -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 [EMAIL PROTECTED] http://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
On Sun, 08 Apr 2007 23:16:18 -0400, "David H. Lynch Jr" <[EMAIL PROTECTED]> said: > [EMAIL PROTECTED] wrote: > > Christer Weinigel: Until YOU, have actually used the REISER4 filesystem > > yourself, I think YOU OWE IT to the people on the linux-kernel mailing > > list, to, AS YOU SAY, shut the fuck up. > > > > Even reading up on the REISER4 filesystem would help. > > > > Applying a little intelligence would undoubtedly help too. > > > John; >I would like to see Reiser4 get into the Kernel too. But angry > rhetoric ... is not going to do it. > I WAS JUST POINTING OUT THE VERY ANGRY RHETORIC COMING FROM CERTAIN FOLK ON THE MAILING LIST. I think it really needs to be pointed out as some of those here are extremely rude, bordering on unpleasant. I only use angry rhetoric to make a point. HAS THAT POINT BEEN MADE? -- [EMAIL PROTECTED] -- http://www.fastmail.fm - Same, same, but differentÂ… - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
Christer Weinigel wrote: > > To misquote Scott Adams: I'm not anti-Reiser4, I'm anti-idiot. > > /Christer > So can we ignore the benchmarks issue entirely and deal with the issue of what might it actually take now that Hans is not pissing all over everyone to get Reiser4 submitted with some hope of being accepted ? Your analysis was excellent. It make the case that just is there will be a few cases where Reiser4 is absolutely compelling and a few cases where it makes no sense at all that there are alot of cases in the middle. I can keep coming up with examples where the attributes unique to Reiser4 are an excellent idea, and you can just as easily come up with cases where it is a bad idea. If the standard is that it must be all things to all people, then we need to quit letting any filesystems in. Let's say John takes your advice and tweaks bonnie or builds whatever benchmark test bed you want, and proves Reiser4 outperforms everything in all cases (highly unlikely) does that mean it should get it ? What about if it is 10% slower in most normal cases ? Is that a good enough reason for excluding it ? Is anyone ready to claim that there are no cases (beyond the extremely rare case of compressing zero's) where Reiser4 may prove to be the best choice ? Compression itself is almost a religious war subject. for some people it is ALWAYS a bad idea, to others it is ALWAYS a good idea. In the real world it varies. Application level compression is generally superior to filesystem compression - there is more knowledge about the data leading to better choice of compression algorithms. and fortunately Linux provides one of the best environments for incorporating application level compression. But generally is not the same as ALWAYS. While I raised a specific case I was aware of where my understanding was compressed data could result in a net overall gain in performance - network servers where compression decompression were handled at the client, several other instances have been raised. Depending on the speed of the CPU, size of memory, size and speed of cache, speed of disk, type of data, sometimes compressed data will not only save space but improve performance - and sometimes it will be costly to performance. That does not make it a bad idea. We have several scheduling algorithms, because one size does not fit all. Besides I doubt given the complexity of the issue that John or anyone else can put together a benchmark that I can't poke holes in. There are so many cases that there is no such thing as the general case. Even performance compressing zeros starts to almost look like a rational measure when you start to calculate all the permutations in the data compression matrix. -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 [EMAIL PROTECTED] http://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
[EMAIL PROTECTED] wrote: > Christer Weinigel: Until YOU, have actually used the REISER4 filesystem > yourself, I think YOU OWE IT to the people on the linux-kernel mailing > list, to, AS YOU SAY, shut the fuck up. > > Even reading up on the REISER4 filesystem would help. > > Applying a little intelligence would undoubtedly help too. > John; I would like to see Reiser4 get into the Kernel too. But angry rhetoric ... is not going to do it. If it was enough Hans did a much better job of insulting the naysayers without resulting to vulgarity, And it got him pretty much nowhere. It is my personal oppinion that the reason Reiser4 did not reach the kernel long ago was because Hans pissed all over way to many people. Making this about people will serve no one. We can all call each other assholes, go home feeling vindicated, but Reiser4 will be no closer to the kernel, and any potential that anyone might have benefited from that will just be pissed away. After that the statistics you are reporting are not TOTALLY meaningless. But they are also not compelling. It is not wise to be arguing benchmarks anyway. I hope that few on this list are not willing to conceed that Reiser4 is fast. Whether it is 10% slower than the best or 3 times as fast is probably not relevant to whether it gets included. All the worst case scenarious of Reiser4 performance and behavior should all still be sufficient to justify letting it in as Experimental. You can argue your benchmarks forever. Your benchmark has narrow applicability - pretty much every benchmark will. And all you do is provoke a firestorm of debate over whether compressing zeros means anything, whether compression is even a good idea, whether, .. The answer to ALL of these questions will be "it depends" I have dealt with huge data sets that were mostly zero's, a file system that had a factor of 3 space savings and or performance benefit might have mattered alot there - but they are not the norm. I have dealt with instances where compression increased performance rather than decreased it. Every "feature" of Reiser4 is an asset in some environments and a liability in others. Whether it is suitable for most environments is nto going to be established by a benchmark. -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 [EMAIL PROTECTED] http://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
David H. Lynch Jr wrote: Jeff Garzik wrote: If the compelling reason is that it needs a test, I'd say its not ready. Can you please elaborate ? I am not sure I understand what you are arguing ? Despite his substantially less than polite rhetoric, I have read Hans's post from months if not years ago. Aside from the pissing contests - which where not entirely one sided, I actually beleive that Hans made a reasonable case that Reiser4 had gone about as far as it could reasonably go with regard to testing, robustness, ... without the broader base of use that even an experimental filesystem in distribution tree would get. I for one would atleast play with it if it were in the distribution tree. As far as I could tell pretty much everything else that was demanded Hans eventually caved and provided - albeit with much pissing and moaning, and holy than thou rhetoric. The argument that anything that needs testing can't get into the distribution tree's is specious. There is alot of poorly tested crap in the distribution trees. I'm arguing against circular logic: the claim that one cannot determine reiser4's true usefulness unless its in the tree. The better method is to get a distro to add reiser4, _then_ if it proves worthy add it to the kernel tree. Not the other way around. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Jeff Garzik wrote: > > If the compelling reason is that it needs a test, I'd say its not ready. Can you please elaborate ? I am not sure I understand what you are arguing ? Despite his substantially less than polite rhetoric, I have read Hans's post from months if not years ago. Aside from the pissing contests - which where not entirely one sided, I actually beleive that Hans made a reasonable case that Reiser4 had gone about as far as it could reasonably go with regard to testing, robustness, ... without the broader base of use that even an experimental filesystem in distribution tree would get. I for one would atleast play with it if it were in the distribution tree. As far as I could tell pretty much everything else that was demanded Hans eventually caved and provided - albeit with much pissing and moaning, and holy than thou rhetoric. The argument that anything that needs testing can't get into the distribution tree's is specious. There is alot of poorly tested crap in the distribution trees. But separately, there is the issue of scale. Namesys claims that they have no currently know bugs, faults ... - with their base of internal and external users. I would fully expect new failures to crop up with any filesystem, driver, ... moving up an order of magnitude in users. Are you going to subject all filesystems and drivers to the same high standards you are placing on Reiser4 ? If so then we need to strip the distribution tree now. I am not looking to defend Hans - he is likely to be in jail and no longer a factor for a long time. Nor am I looking to make or support claims for Reiser4. But I am asking - why we can not get past the bad blood, rhetoric, and zealotry -which to my eyes has not been all one sided. I am NOT looking for a technical explanation of all the relative merits and demerits of Reiser4. I do not care for arguments about whether it compresses 0's well, or that tail combining is a bad thing. They may have merit, but there is not a filesystem that is going to be all things to all people. Whether Reiser4 is a small niche filesystem or a significant general use one, is a decision that should be reached by its performance in practice, not it rhetoric. Regardless, even as a niche filesystem, I beleive at this point it merits inclusion. -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 [EMAIL PROTECTED] http://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
[linux-kernel trimmed from the Cc list] [EMAIL PROTECTED] writes: > [lots of drivel with lots of capital letters elided] > [a totally confused mess of responses to at least three different mails] Wow. You are absolutely amazing. But, oh, well. > By the way: Did I thank you "delightful" people for the "pleasant" > welcome to the linux-kernel mailing list? You are so welcome. *grin* > > So the two bonnie benchmarks with lzo and gzip are > > totally meaningless for any real life usages. > > YOU (yes, the one with no experience and next to NO knowledge on the > subject) claim that because bonnie++ writes files that are mostly zeros, > the results are meaningless. It should be mentioned that bonnie++ writes > files that are mostly zero for all the filesystems compared. So the > results are meaningful, contrary to would you claim. Ok, lets take this really slowly so that you may understand. Compression in the file system can be useful, there is no doubt about that, but you have to be aware of the tradeoffs. First of all, very few people have any use for storing files consisting of just zeroes. Trying to make any decision based on a file systems ability to compress zeroes is just plain dumb. Bonnie++ assumes that the data it writes will end up being written to disk and not be compressed. Right now it allocates a buffer which is filled with zeroes and half a dozen bytes that the beginning of the buffer are filled in with some random data. So to make the bonnie runs have any meaning on any compressed file systems you really want to be able to choose what data it writes to disk. If you modified bonnie to do multiple test runs, one with zeroes, another one with some easily compressed data such as a syslog, some not so easily compressed data such as the contents of /bin/bash and some uncompressible data such as from /dev/urandom that would be a lot better benchmark. Then you have to be aware of the cost of that compression. First of all, it is going to use some CPU, so measuring the CPU load during the benchmark is a good start. Another thing with compression is that it requires you to keep both some compressed and the uncompressed data in RAM at the same time, so the memory pressure will increase. This is harder to measure and quantify. Finally, since the CPU has to get involved at compressing and decompressing the data doing this will pull both the uncompressed and compressed data into the CPU cache and may evict data from the cache that other processes would have found useful. This cache pollution is even harder to measure. And none of these costs make any difference for benchmarks run on a lightly loaded system, but may make a difference in real life on any system that tries to do something useful at the same time. Then you have to consider your use cases. As I said in my previous mail, for my only space constrained disk, I store a lot of large flac encoded CD images. That data is basically uncompressible, so compression buys me nothing, it just costs me a lot of extra CPU to try to compress uncompressible data. In addition, each CD image is a quite large too, about 300MByte for a full size album, so whatever savings I can get though the tail merging that Reiser4 (and Reiser3) does is marginal for my use case. Other use cases might have a lot to gain from compression. > ALSO YOU IGNORE examples offered by others, on lkml, which contradict > your assertion: FOR EXAMPLE: > > > I see the same thing with my nightly scripts that do syslog > > analysis, last year I trimmed 2 hours from the nightly run by > > processing compressed files instead of uncompressed ones (after I > > did this I configured it to compress the files as they are rolled, > > but rolling every 5 min the compression takes <20 seconds, so the > > compression is < 30 min) > > David has said that compressing the logs takes > > 24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day) > > but he saves 2 hours of CPU time on the daily syslog analysis. > > For a total (minimum) saving of 24 minutes. So lets look at the syslog case then. First of all, lets compress my syslog with gzip: gzip -c /var/log/messages >whole.gz du -h /var/log/messages whole.gz 532Kmessages 64K whole.gz Unfortunately, this compressed format isn't very efficient for some use cases, lets say that I want to read the last 10 lines of the syslog. On a normal uncompressed file system I can just seek to the end of the file, read the last block and get those 10 lines (or if the last block didn't have 10 lines, I can try the block before that). But with a compressed file, I have to uncompress the whole file and throw away 531 kBytes at the begining of the file to do that. So a file system that wants to give the user efficient random access to a files can't compress the whole file as done above. It has to make some tradeoffs to make random access practically usable. Most compressing file systems do that by spl
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
Wow, I'm impressed. Think you got the record on how many mails you referenced to in a reply... But dude, please calm down, the caps-lock is not the answer. You have got some rude answers and you have called them back on it + you have repeated the same statement several times, that is not the best way of convincing people. I believe you picked up the "anti-Reiser religion"-phrase from previous rant-wars (otherwise, why does that "religion"-phrase always come up, and (almost) only when dealing with Reiser-fs), and yes, there has been some clashes caused by both sides, so please be careful when dealing with this matter. Would you be willing to benchmark Reiser4 with some compressed binary-blob and show the time as well as the CPU-usage? And document how it is set up so it can be reproduced. After all, Windows is suppose to be more stable, maintained and cost-efficient then Linux, but they don't tell us how ;) since it can't benefit as much from similarity between files. So if that is the case and you really want to save diskspace you almost have to look at read-only compressed filesystems such as cramfs, squashfs, zisofs, cloop and various other variants in combination with a unionfs overlay to get read/write functionality. But in the end everything is a tradeoff. You can save diskspace, but increase the cost of corruption. You deliberately ignored the fact that bad blocks are NOT dealt with by the filesystem,... but by the operating system. Like I said: If your filesystem is writing to bad blocks, then throw away your operating system. I may have missed something, but if my room-mate took my harddrive, screwed it open, wrote a love-letter on the disk with a pencil and then returned it (ok, there may be some more plausible reasons for corruption), is the OS really suppose to handle it? Yes, it should not assign any new data to those blocks but should it not also fall into the file-systems domain to be able to restore some/all data? Just my 2c to the pond Richard Knutsson - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel
Christer Weinigel: Until YOU, have actually used the REISER4 filesystem yourself, I think YOU OWE IT to the people on the linux-kernel mailing list, to, AS YOU SAY, shut the fuck up. Even reading up on the REISER4 filesystem would help. Applying a little intelligence would undoubtedly help too. > [EMAIL PROTECTED] writes: > > > Lennart. Tell me again that these results from > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > are not of interest to you. I still don't understand why you > > have your head in the sand. > > Oh, for fucks sake, stop sounding like a broken record. Oh, for fucks sake, would you, and your religious anti-REISER cohorts, stop sounding like a broken record. > You have repeated the same totally meaningless statistics more > times than I care to count. Please shut the fuck up. You, and your religious anti-REISER cohorts, have indeed repeated the same broken arguments (if you can call them such) more times than I care to count. NO statistics, NO real facts, just selective MANIPULATION of facts. > Please shut the fuck up. Yes, why don't you politely, shut the fuck up. Until YOU, have actually used the REISER4 filesystem yourself, I think YOU OWE IT to the people on the linux-kernel mailing list, to shut the fuck up, as YOU say. I guess, the fact that you are a TOTAL HYPOCRITE, has completely escaped you. By the way: Did I thank you "delightful" people for the "pleasant" welcome to the linux-kernel mailing list? - > So the two bonnie benchmarks with lzo and gzip are > totally meaningless for any real life usages. YOU (yes, the one with no experience and next to NO knowledge on the subject) claim that because bonnie++ writes files that are mostly zeros, the results are meaningless. It should be mentioned that bonnie++ writes files that are mostly zero for all the filesystems compared. So the results are meaningful, contrary to would you claim. And hopefully all will notice that you just ignore these tests: .-. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32| 988 | 253 | 158 | 118 | 81 | 95 | .-. where the files are definitely NOT mostly zeros. Your negligence has to be deliberate,... but why? Are you manipulating the facts just to try and win an argument? Most sane people will realize, that what you say is simply wrong. ALSO YOU IGNORE examples offered by others, on lkml, which contradict your assertion: FOR EXAMPLE: > I see the same thing with my nightly scripts that do syslog analysis, last > year > I trimmed 2 hours from the nightly run by processing compressed files instead > of > uncompressed ones (after I did this I configured it to compress the files as > they are rolled, but rolling every 5 min the compression takes <20 seconds, > so > the compression is < 30 min) >From David Lang http://lkml.org/lkml/2007/4/7/196 Willy Tarreau also mentions this situation in a couple of articles. Let me spoon feed you: David has said that compressing the logs takes 24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day) but he saves 2 hours of CPU time on the daily syslog analysis. For a total (minimum) saving of 24 minutes. The actual saving is probably much greater. It depends on the CPU utilization when not compressing, ie, whether you are using ide CPU cycles or not. I guess it also depends on whether you can go home one and a half hours earlier by using compression, or if your boss makes you stick around anyway. NOTE THAT THE FILES IN THIS EXAMPLE ARE ALSO NOT MAINLY ZEROS. MAYBE you just lacked the knowledge to understand what David was saying, or maybe your desire to denigrate REISER4 is so strong, that you simply don't care what other people say about similar circumstances. I am not sure why you have to be spoon feed on these matters, or why you adamantly refuse to find the facts of the matter, for yourself. --
Re: Reiser4. BEST FILESYSTEM EVER.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Theodore Tso wrote: > The reason why I ignore the tar+gzip tests is that in the past Hans > has rigged the test by using a tar ball which was generated by > unpacking a set of kernel sources on a reiser4 filesystem, and then > repacking them using tar+gzip. The result was a tar file whose files > were optimally laid out so that reiser4 could insert them into the > filesystem b-tree without doing any extra work. > > I can't say for sure whether or not this set of benchmarks has done > this (there's not enough information describing the benchmark setup), > but the sad fact of the matter is that people trying to pitch Reiser4 > have generated for themselves a reputation for using rigged > benchmarks. Hans's used of a carefully stacked and ordered tar file > (which is the same as stacking a deck of cards), and your repeated use > of the bonnee++ benchmarks despite being told that it is a meaningless > result given the fact that well, zero's compress very well and most > people are interested in storing a file of all zeros, has caused me to > look at any benchmarks cited by Reiser4 partisans with a very > jaundiced and skeptical eye. > > Fortunately for you, it's not up to me whether or not Reiser4 makes it > into the kernel. And if it works for you, hey, go wild. You can > always patch it into your own kernel and encourage others to do the > same with respect to getting it tested and adopted. My personal take > on it is that Reiser3, Reiser4 and JFS suffer the same problems, which > is to say they have a very small and limited development community, > and this was referenced in Novell's decision to drop Reiser3: > > http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/ > > SuSE has deprecated Reiser3 *and* JFS, and I believe quite strongly it > is the failure of the organizations to attract a diverse development > community is ultimately what doomed them in the long term, both in > terms of support as the kernel migrated and new feature support. It > is for that reason that Hans' personality traits that tend to drive > away those developers who would help them, beyond those that he hires, > is what has been so self-destructive to Reiser4. Read the > announcement Jeff Mahoney from SUSE Labs again; he pointed out was > that reiser3 was getting dropped even though it performs better than > ext3 in some scenarios. There are many other considerations, such as > a filesystem's robustness in case on-disk corruption, long term > maintenance as the kernel maintains, availability of developers to > provide bug fixes, how well the system performs on systems with > multiple cores/CPU's, etc. Those are all arguments I've made and still stand by, but I should address one point that has been repeated fairly often. Novell _isn't_ dropping support for Reiser3 in any of our products. The change only refers to the choice of a default file system. Most users don't care about which file system they use, and those that do are still free to choose reiser3 if they want it. We'll support it and I still have patches under development to improve it. - -Jeff - -- Jeff Mahoney SUSE Labs -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGGTHYLPWxlyuTD7IRAj0SAJ4txD5NoStOA4GFgkzcXDdE/Xf9ngCZATNL QtyNTGbi6YFbNF71T5C9hTA= =Emwr -END PGP SIGNATURE- - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
The reason why I ignore the tar+gzip tests is that in the past Hans has rigged the test by using a tar ball which was generated by unpacking a set of kernel sources on a reiser4 filesystem, and then repacking them using tar+gzip. The result was a tar file whose files were optimally laid out so that reiser4 could insert them into the filesystem b-tree without doing any extra work. I can't say for sure whether or not this set of benchmarks has done this (there's not enough information describing the benchmark setup), but the sad fact of the matter is that people trying to pitch Reiser4 have generated for themselves a reputation for using rigged benchmarks. Hans's used of a carefully stacked and ordered tar file (which is the same as stacking a deck of cards), and your repeated use of the bonnee++ benchmarks despite being told that it is a meaningless result given the fact that well, zero's compress very well and most people are interested in storing a file of all zeros, has caused me to look at any benchmarks cited by Reiser4 partisans with a very jaundiced and skeptical eye. Fortunately for you, it's not up to me whether or not Reiser4 makes it into the kernel. And if it works for you, hey, go wild. You can always patch it into your own kernel and encourage others to do the same with respect to getting it tested and adopted. My personal take on it is that Reiser3, Reiser4 and JFS suffer the same problems, which is to say they have a very small and limited development community, and this was referenced in Novell's decision to drop Reiser3: http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/ SuSE has deprecated Reiser3 *and* JFS, and I believe quite strongly it is the failure of the organizations to attract a diverse development community is ultimately what doomed them in the long term, both in terms of support as the kernel migrated and new feature support. It is for that reason that Hans' personality traits that tend to drive away those developers who would help them, beyond those that he hires, is what has been so self-destructive to Reiser4. Read the announcement Jeff Mahoney from SUSE Labs again; he pointed out was that reiser3 was getting dropped even though it performs better than ext3 in some scenarios. There are many other considerations, such as a filesystem's robustness in case on-disk corruption, long term maintenance as the kernel maintains, availability of developers to provide bug fixes, how well the system performs on systems with multiple cores/CPU's, etc. - Ted - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Sat, Apr 07, 2007 at 01:10:31PM -0400, [EMAIL PROTECTED] wrote: > On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said: > > > > Think about it,... read speeds that are some FOUR times the physical > > > disk read rate,... impossible without the use of compression (or > > > something similar). > > > > It's really impossible with compression only unless you're writing > > only zeros or stuff alike. I don't know what bonnie uses for testing > > but real life data doesn't compress 4 times. Two times, sometimes, > > All depends on your data. From a recent "compress the old logs" job on > our syslog server: > > /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308: 85.4% -- replaced > with /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308.gz > > And it wasn't a tiny file either - it's a busy mailserver, the logs run to > several hundred megabytes a day. Syslogs *often* compress 90% or more, > meaning a 10X compression. > > > but then it will be typically slower than disk access (I mean read, > > as write will be much slower). > > Actually, as far back as 1998 or so, I was able to document 20% *speedups* > on an AIX system that supported compressed file systems - and that was from > when a 133mz PowerPC 604e was a *fast* machine. Since then, CPUs have gotten > faster at a faster rate than disks have, even increasing the speedup. > > The basic theory is that unless you're sitting close to 100%CPU, it is > *faster* > to burn some CPU to compress/decompress a 4K chunk of data down to 2K, and > then > move 2K to the disk drive, than it is to move 4K. It's particularly noticable > for larger files - if you can apply the compression to remove the need to > move > 2M of data faster than you can move 2M of data, you win. Counterpoints: - not only CPUs have became faster, RAM has become faster, too a kernel tree after an allyesconfig build is at about 1 GB which is less than half the size of RAM in my desktop computer if all disk accesses are asynchronous write accesses without any pressure of being done quickly, compression can't improve performance - today, much of the bigger data is already compressed data like mp3s or movies - for cases like logfiles or databases, application specific compression should give best results There might be special cases where compressed filesystems make sense, but my impression is that filesystem compresssion is not important and suited for current average systems. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Sun, Apr 08, 2007 at 06:21:29AM -0700, [EMAIL PROTECTED] wrote: > Jose, > since you clearly have nothing useful to say. Why don't you let Teddy > talk for himself. John, You should first apply your own advice to yourself. Annoying everyone with the exact same mail 10 times a day is really disserving to the cause you pretend to defend. Please stop tainting reiser4's reputation, because I suspect that it can do far more things than what you make it look like. Its developers certainly need useful reports instead of a mentally deficient's rant. Now please call the nurse for your injection and go back to bed. Thank you in advance, Willy - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Jose, since you clearly have nothing useful to say. Why don't you let Teddy talk for himself. On Sun, 8 Apr 2007 13:48:11 +0100, "Jose Celestino" <[EMAIL PROTECTED]> said: > Words by [EMAIL PROTECTED] [Sat, Apr 07, 2007 at 09:13:32PM > -0700]: > > Teddy, > > > > It is a pity you don't address the full set of results, when you make > > your snide comments. > > > > Now since you have them,... why don't you make reasoned comment about > > them. > > > > You can read more here: > > > > John, > > it is not because you keep posting the same numbers over and over again > (or is this your new signature?) that they will be more substantiated > (hint: cpu usage). > Just more annoying each time. > > I'll remember to use reiser4 for my > 90-percent-zero-files-no-need-for-proven-robustness-and-plenty-of-cpu-to-spare > boxes. Thank you. > > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > .-. > > | FILESYSTEM | TIME |DISK | > > | TYPE |(secs)|USAGE| > > .-. > > |REISER4 lzo | 1938 | 278 | > > |REISER4 gzip| 2295 | 213 | > > |REISER4 | 3462 | 692 | > > |EXT2| 4092 | 816 | > > |JFS | 4225 | 806 | > > |EXT4| 4408 | 816 | > > |EXT3| 4421 | 816 | > > |XFS | 4625 | 779 | > > |REISER3 | 6178 | 793 | > > |FAT32 |12342 | 988 | > > |NTFS-3g |10414 | 772 | > > .-. > > > > > > Column one measures the time taken to complete the bonnie++ benchmarking > > test (run with the parameters bonnie++ -n128:128k:0) > > > > Column two, Disk Usage: measures the amount of disk used to store 655MB > > of raw data (which was 3 different copies of the Linux kernel sources). > > > > OR LOOK AT THE FULL RESULTS: > > > > .-. > > |File |Disk |Copy |Copy |Tar |Unzip| Del | > > |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | > > |Type | (MB)| (1) | (2) |655MB|655MB| Gig | > > .-. > > |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | > > |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | > > |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | > > |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | > > |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | > > |NTFS | 779 | 781 | 173 | X | X | X | > > |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | > > |XFS | 799 | 220 | 173 | 119 | 90 | 106 | > > |JFS | 806 | 228 | 202 | 95 | 97 | 127 | > > |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | > > |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | > > |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | > > |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | > > |FAT32| 988 | 253 | 158 | 118 | 81 | 95 | > > .-. > > > > > > Each test was preformed 5 times and the average value recorded. > > Disk Usage: The amount of disk used to store the data (which was 3 > > different copies of the Linux kernel sources). > > The raw data (without filesystem meta-data, block alignment wastage, > > etc) was 655MB. > > Copy 655MB (1): Copy the data over a partition boundary. > > Copy 655MB (2): Copy the data within a partition. > > Tar Gzip 655MB: Tar and Gzip the data. > > Unzip UnTar 655MB: UnGzip and UnTar the data. > > Del 2.5 Gig: Delete everything just written (about 2.5 Gig). > > > > > > To get a feel for the performance increases that can be achieved by > > using compression, we look at the total time (in seconds) to run the > > test: > > > > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) > > > > .---. > > | FILESYSTEM | TIME | > > .---. > > |REISER4 lzo | 1938| > > |REISER4 gzip| 2295| > > |REISER4 | 3462| > > |EXT4| 4408| > > |EXT2| 4092| > > |JFS | 4225| > > |EXT3| 4421| > > |XFS | 4625| > > |REISER3 | 6178| > > |FAT32 | 12342| > > |NTFS-3g |>10414| > > .---. > > > > > > > > On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <[EMAIL PROTECTED]> said: > > > On Sat, Apr 07, 2007 at 05:44:57PM -0700, [EMAIL PROTECTED] > > > wrote: > > > > To get a feel for the performance increases that can be achieved by > > > > using compression, we look at the total time (in seconds) to run the > > > > test: > > > > > > You mean the performance increases of writing a file which is mostly > > > all zero's? Yawn. > > > > > > - Ted > > -- > > > > [EMAIL PROTECTED] > > > > -- > > http://www.fastmail.fm - IMAP accessible web-mail > > > > - > > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > > the body of a message to [EMAIL PROTECTED] > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > Jose Celestino > > htt
Re: Reiser4. BEST FILESYSTEM EVER.
Words by [EMAIL PROTECTED] [Sat, Apr 07, 2007 at 09:13:32PM -0700]: > Teddy, > > It is a pity you don't address the full set of results, when you make > your snide comments. > > Now since you have them,... why don't you make reasoned comment about > them. > > You can read more here: > John, it is not because you keep posting the same numbers over and over again (or is this your new signature?) that they will be more substantiated (hint: cpu usage). Just more annoying each time. I'll remember to use reiser4 for my 90-percent-zero-files-no-need-for-proven-robustness-and-plenty-of-cpu-to-spare boxes. Thank you. > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > .-. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2| 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4| 4408 | 816 | > |EXT3| 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-. > > > Column one measures the time taken to complete the bonnie++ benchmarking > test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB > of raw data (which was 3 different copies of the Linux kernel sources). > > OR LOOK AT THE FULL RESULTS: > > .-. > |File |Disk |Copy |Copy |Tar |Unzip| Del | > |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | > |Type | (MB)| (1) | (2) |655MB|655MB| Gig | > .-. > |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | > |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | > |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | > |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | > |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | > |NTFS | 779 | 781 | 173 | X | X | X | > |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | > |XFS | 799 | 220 | 173 | 119 | 90 | 106 | > |JFS | 806 | 228 | 202 | 95 | 97 | 127 | > |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | > |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | > |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | > |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | > |FAT32| 988 | 253 | 158 | 118 | 81 | 95 | > .-. > > > Each test was preformed 5 times and the average value recorded. > Disk Usage: The amount of disk used to store the data (which was 3 > different copies of the Linux kernel sources). > The raw data (without filesystem meta-data, block alignment wastage, > etc) was 655MB. > Copy 655MB (1): Copy the data over a partition boundary. > Copy 655MB (2): Copy the data within a partition. > Tar Gzip 655MB: Tar and Gzip the data. > Unzip UnTar 655MB: UnGzip and UnTar the data. > Del 2.5 Gig: Delete everything just written (about 2.5 Gig). > > > To get a feel for the performance increases that can be achieved by > using compression, we look at the total time (in seconds) to run the > test: > > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) > > .---. > | FILESYSTEM | TIME | > .---. > |REISER4 lzo | 1938| > |REISER4 gzip| 2295| > |REISER4 | 3462| > |EXT4| 4408| > |EXT2| 4092| > |JFS | 4225| > |EXT3| 4421| > |XFS | 4625| > |REISER3 | 6178| > |FAT32 | 12342| > |NTFS-3g |>10414| > .---. > > > > On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <[EMAIL PROTECTED]> said: > > On Sat, Apr 07, 2007 at 05:44:57PM -0700, [EMAIL PROTECTED] > > wrote: > > > To get a feel for the performance increases that can be achieved by > > > using compression, we look at the total time (in seconds) to run the > > > test: > > > > You mean the performance increases of writing a file which is mostly > > all zero's? Yawn. > > > > - Ted > -- > > [EMAIL PROTECTED] > > -- > http://www.fastmail.fm - IMAP accessible web-mail > > - > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Jose Celestino http://www.msversus.org/ ; http://techp.org/petition/show/1 http://www.vinc17.org/noswpat.en.html "And on the trillionth day, Man created Gods." -- Thomas D. Pate - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
David H. Lynch Jr wrote: I do care about getting Reiser4 into the kernel so that it can actually get a real test, and frankly do not see any compelling reason that should not happen. If the compelling reason is that it needs a test, I'd say its not ready. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
[EMAIL PROTECTED] writes: > Lennart. Tell me again that these results from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > are not of interest to you. I still don't understand why you have your > head in the sand. Oh, for fucks sake, stop sounding like a broken record. You have repeated the same totally meaningless statistics more times than I care to count. Please shut the fuck up. As you discovered yourself (even though you seem to fail to understand the significance of your discovery), bonnie writes files that consist of mostly zeroes. If your normal use cases consist of creating a bunch of files containing zeroes, reiser4 with compression will do great. Just lovely. Except that nobody sane would store a lot of files containing zeroes, except an an excercize in mental masturbation. So the two bonnie benchmarks with lzo and gzip are totally meaningless for any real life usages. As for the amount of disk needed to store three kernel trees, the figures you quote show that Reiser4 does tail combining where the tail of multiple files are stored in one disk block. A nice trick that seems save you about 15% disk space compared to ext3. Now you have to realise what that means, it means that if the disk block containing those tails (or any metadata pointing at that block) gets corrupted, instead of just losing one disk block for one file, you will have lost the tail for all the files sharing that disk block. Depending on your personal prioritites, saving 15% of the space may be worth the risk to you, or maybe not. Personally, for the only disk I'm short on space on, I mostly store flac encoded images of my CD collection, and saving 2kByte out of every 300MByte disk simply doesn't make any difference, and I much prefer a stable file system that I can trust not to lose my data. You might make different choices. The same goes for just about every feature that you tout, it has its advantages, and it has its disadvantages. Doing compression on data is great if the data you store is compressible, and sucks if it isn't. Doing compression on each disk block and then packing multiple compressed blocks into each physical disk block will probably save some space if the data is compressible, but at the same time it means that you will spend a lot of CPU (and cache footprint) compressing and uncompressing that data. On a single user system where the CPU is mostly idle it might not make much of a difference, on a heavily loaded multiuser system it might do. Logs can be compressed quite well using a block based compression scheme, but the logs can be compressed even better by doing compression on the whole file with gzip. So what's the best choice, to do transparent compression on the fly giving ok compression or teaching the userspace tools to do compression of old logs and get really good compression? Or maybe disk space really isn't that important anyway and the best thing is to just leave the logs uncompressed. Another example: one of the things Reiser3 is supposed to be really good for is storing an INN news spool, doing tail merging of lots of individual files containing articles gives a great space saving, and since it's just a news spool, reliability in face of a system crashes really don't matter all that much. On the other hand, INN's Cyclic News File System running on top of ext2 is probably an even better choice in that case. What do you want to use? What I want to get at is that you can troll the mailing lists (and crossposting stupid inflammatory material with an inane subject to a bunch of mailing lists the way you have done definitely is trolling) trying to say that whatever you're trying to sell is the best, but at the end, if a file system is better or not is a lot more complex than quoting just one benchmark (which, once again, is meaningless, compressing a lot of zeroes is simple and really does not tell you anything about real world usages). And there are other considerations too, even if Reiser4 would be the best thing since sliced breadd, can I trust Hans Reiser to support Reiser4 for the next five years? Or will he drop support for Reiser4 the same way he dropped support for the old Reiser3 when Reiser4 came along? Or will he drop Reiser4 when the grant to do Reiser 4 development expires? /Christer -- "Just how much can I get away with and still go to heaven?" Christer Weinigel <[EMAIL PROTECTED]> http://www.weinigel.se - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
[EMAIL PROTECTED] wrote: > > (And for those who talk about backups - yes, taking backups is good. > However, it's the rare laptop or desktop machine that can afford the > luxury of RAID disks, and backups usually happen once a night, if that > often. This means that if you've been working hard on something important > all day, and the disk blows chunks at 4:30PM, you *will* be suddenly very > concerned over exactly how much you can recover off the failing drive Been there done, that. I am not sure that RAID is physically practical for a laptop, but it may be substantially more practical and economical for a desktop or home server, than the hope that a home user might backup a system. I am sure that there are plenty of people with true stories of backup systems saving their lives. But my personal history involves systems with no backup, systems with poor backup, systems with wishful backup, and finally systems with failed backup. My first experience with tape drivers was disassembling and patching a QIC driver (not Linux) because small portions of the backup tape were unreadable (the client smoked while backing the system up) and the tape software thought recovery was an all or nothing proposition. My first personal rule of data disaster recovery is start with the data on the disk. I have only had rare instances where 99% of the data on a disk was not readable and recoverable - though for some reason the most critical data is always in that 1% that puts up a fight. Backup systems are what you go to after you have recovered everything possible from the disk. -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 [EMAIL PROTECTED] http://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Teddy, It is a pity you don't address the full set of results, when you make your snide comments. Now since you have them,... why don't you make reasoned comment about them. You can read more here: http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm .-. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2| 4092 | 816 | |JFS | 4225 | 806 | |EXT4| 4408 | 816 | |EXT3| 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). OR LOOK AT THE FULL RESULTS: .-. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32| 988 | 253 | 158 | 118 | 81 | 95 | .-. Each test was preformed 5 times and the average value recorded. Disk Usage: The amount of disk used to store the data (which was 3 different copies of the Linux kernel sources). The raw data (without filesystem meta-data, block alignment wastage, etc) was 655MB. Copy 655MB (1): Copy the data over a partition boundary. Copy 655MB (2): Copy the data within a partition. Tar Gzip 655MB: Tar and Gzip the data. Unzip UnTar 655MB: UnGzip and UnTar the data. Del 2.5 Gig: Delete everything just written (about 2.5 Gig). To get a feel for the performance increases that can be achieved by using compression, we look at the total time (in seconds) to run the test: bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) .---. | FILESYSTEM | TIME | .---. |REISER4 lzo | 1938| |REISER4 gzip| 2295| |REISER4 | 3462| |EXT4| 4408| |EXT2| 4092| |JFS | 4225| |EXT3| 4421| |XFS | 4625| |REISER3 | 6178| |FAT32 | 12342| |NTFS-3g |>10414| .---. On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <[EMAIL PROTECTED]> said: > On Sat, Apr 07, 2007 at 05:44:57PM -0700, [EMAIL PROTECTED] > wrote: > > To get a feel for the performance increases that can be achieved by > > using compression, we look at the total time (in seconds) to run the > > test: > > You mean the performance increases of writing a file which is mostly > all zero's? Yawn. > > - Ted -- [EMAIL PROTECTED] -- http://www.fastmail.fm - IMAP accessible web-mail - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Lennart Sorensen wrote: > Nope, if they are accurate and they have something to do with your > particular usage and applications, then they are relevant. But it > requires both to make them relevant. Although it may be possible for a > benchmark to be relevant even if not particularly accurate. I do not care about the benchmarks or the compression. They are pointless anyway, Reiser4 is not and can not be all things to all people. I do not care how many angels can dance on the head of a pin. Whether it is most suitable for broad use or only narrow cases remains to seen. Hans Reiser is now in jail awaiting trial for the murder of his wife - I doubt he is going to be posting and stirring things up anytime soon Those who need to assurance of their moral superiority can take that as proof if they need. I do care about getting Reiser4 into the kernel so that it can actually get a real test, and frankly do not see any compelling reason that should not happen. Their may be better filesystems currently in the kernel - there certainly are crappier ones. I have used ReiserFs extensively, and on the whole been extremely happy with it. I have been slightly singed - a long time ago, but it is certainly not the only file system that has caused me grief. I can not think of an existing Linux (or other) filesystem that is the be all or end all. I have no time for either the zealots that think Reiser4 is the next best thing to sliced bread nor those who are going to find some case to demonstrate it is total crap - no matter what. Most people are wise not to use Reiser4 - or any other new filesystem until it has been rigorously tested. I have no bone to pick with that. I will try it once it is available in standard kernel sources, but I am juggling enough patches as it is, and I am not the patch wizard that some of the rest of you are. If I get burned - so be it. It won't be the first time. This game has gone on sufficiently long that Reiser4 has zero hope of becoming more than a footnote used primarily by oddballs and misfits. But there is still some hope that a slightly wider audience might result in numerous positive benefits to other filesystems. I have read many of the critiques - but even if they are valid, I thought Linux was a place where competing ideas were tested in use not rhetoric. -- Dave Lynch DLA Systems Software Development:Embedded Linux 717.627.3770 [EMAIL PROTECTED] http://www.dlasys.net fax: 1.253.369.9244Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Sat, Apr 07, 2007 at 05:44:57PM -0700, [EMAIL PROTECTED] wrote: > To get a feel for the performance increases that can be achieved by > using compression, we look at the total time (in seconds) to run the > test: You mean the performance increases of writing a file which is mostly all zero's? Yawn. - Ted - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Sat, Apr 07, 2007 at 05:44:57PM -0700, [EMAIL PROTECTED] wrote: > Lennart. Tell me again that these results from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm Hmm, copying kernel sources around. Not that interesting. How does it handle mpeg2 files (the majority of my personal data files is on a mythtv system). So a few large files, with mostly linear access, and the occational file deletion. Compression would gain nothing. > are not of interest to you. I still don't understand why you have your > head in the sand. Well I find it hard to get excited about new filesystems. I had sufficiently nasty data loses due to reiserfs 3 back in the early 2.4 kernel days, that I no longer get excited about new filesystems. now I want something I trust that hasn't destroyed any of my data. I tried XFS for a while, ut the early 2.6 kernels had some nasty bugs in XFS too that made that pretty much unusable. Now I just stick with ext3. Screw performance, give me something that works all the time. > .-. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2| 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4| 4408 | 816 | > |EXT3| 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-. > > > Column one measures the time taken to complete the bonnie++ benchmarking > test (run with the parameters bonnie++ -n128:128k:0) Time without cpu usage is not interesting. If you can increase filesystem speed by 10% by doubling cpu load, then I don't want the increase. It is all relative. Wall clock time by itself just doesn't contain enough data to be useful. > Column two, Disk Usage: measures the amount of disk used to store 655MB > of raw data (which was 3 different copies of the Linux kernel sources). I remember disk compression from the DOS days. Disk space is too cheap to bother with that crap anymore. I don't care if it can theoretically turn idle cpu cycles into improved disk speed. Sometimes I don't have idle cpu cycles to waste on that. > OR LOOK AT THE FULL RESULTS: > > .-. > |File |Disk |Copy |Copy |Tar |Unzip| Del | > |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | > |Type | (MB)| (1) | (2) |655MB|655MB| Gig | > .-. > |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | > |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | > |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | > |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | > |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | > |NTFS | 779 | 781 | 173 | X | X | X | > |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | > |XFS | 799 | 220 | 173 | 119 | 90 | 106 | > |JFS | 806 | 228 | 202 | 95 | 97 | 127 | > |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | > |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | > |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | > |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | > |FAT32| 988 | 253 | 158 | 118 | 81 | 95 | > .-. > > > Each test was preformed 5 times and the average value recorded. > Disk Usage: The amount of disk used to store the data (which was 3 > different copies of the Linux kernel sources). > The raw data (without filesystem meta-data, block alignment wastage, > etc) was 655MB. > Copy 655MB (1): Copy the data over a partition boundary. > Copy 655MB (2): Copy the data within a partition. > Tar Gzip 655MB: Tar and Gzip the data. > Unzip UnTar 655MB: UnGzip and UnTar the data. > Del 2.5 Gig: Delete everything just written (about 2.5 Gig). > > To get a feel for the performance increases that can be achieved by > using compression, we look at the total time (in seconds) to run the > test: kernel sources are some of the most compressable data files around. Try with some interesting data instead, like something with larger files, mostly binary, which isn't likely to compess very much. > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) > > .---. > | FILESYSTEM | TIME | > .---. > |REISER4 lzo | 1938| > |REISER4 gzip| 2295| > |REISER4 | 3462| > |EXT4| 4408| > |EXT2| 4092| > |JFS | 4225| > |EXT3| 4421| > |XFS | 4625| > |REISER3 | 6178| > |FAT32 | 12342| > |NTFS-3g |>10414| > .---. > -- Well Reiser4 certainly looks impresive, but I still want to know what the cpu load is like, what the repair tools are like, how well it handles power failures in the middle of a write (I didn't like the way reiser3 would cl
Re: Reiser4. BEST FILESYSTEM EVER.
Lennart. Tell me again that these results from http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm are not of interest to you. I still don't understand why you have your head in the sand. .-. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2| 4092 | 816 | |JFS | 4225 | 806 | |EXT4| 4408 | 816 | |EXT3| 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). OR LOOK AT THE FULL RESULTS: .-. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32| 988 | 253 | 158 | 118 | 81 | 95 | .-. Each test was preformed 5 times and the average value recorded. Disk Usage: The amount of disk used to store the data (which was 3 different copies of the Linux kernel sources). The raw data (without filesystem meta-data, block alignment wastage, etc) was 655MB. Copy 655MB (1): Copy the data over a partition boundary. Copy 655MB (2): Copy the data within a partition. Tar Gzip 655MB: Tar and Gzip the data. Unzip UnTar 655MB: UnGzip and UnTar the data. Del 2.5 Gig: Delete everything just written (about 2.5 Gig). To get a feel for the performance increases that can be achieved by using compression, we look at the total time (in seconds) to run the test: bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) .---. | FILESYSTEM | TIME | .---. |REISER4 lzo | 1938| |REISER4 gzip| 2295| |REISER4 | 3462| |EXT4| 4408| |EXT2| 4092| |JFS | 4225| |EXT3| 4421| |XFS | 4625| |REISER3 | 6178| |FAT32 | 12342| |NTFS-3g |>10414| .---. -- [EMAIL PROTECTED] -- http://www.fastmail.fm - Access all of your messages and folders wherever you are - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
[EMAIL PROTECTED] writes: > I am quite sure that the kernel RPM file is *already* compressed, at least > somewhat. Sure - that's the point - it's better to have the tool compress data when it makes sense. OTOH I think Reiser4 fs is not about transparent compression, it's rather about the plugins etc. There are other filesystems with transparent compression, that's nothing new. -- Krzysztof Halasa - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Thu, Apr 05, 2007 at 09:32:11PM -0700, [EMAIL PROTECTED] wrote: > Don't you agree, that "If they are accurate, THEN they are obviously > very relevant." Nope, if they are accurate and they have something to do with your particular usage and applications, then they are relevant. But it requires both to make them relevant. Although it may be possible for a benchmark to be relevant even if not particularly accurate. > I have set up a Reiser4 partition with gzip compression, here is the > difference in disk usage of a typical Debian installation on two 10GB > partitions, one with Reiser3 and the other with Reiser4. > > debian:/# df > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda3 10490104 6379164 4110940 61% /3 > /dev/sda7 9967960 2632488 7335472 27% /7 > > Partitions 3 and 7 have exactly the same data on them (the typical > Debian install). > > The partitions are exactly the same size (although df records different > sizes). > > Partition 3 is Reiser3 -- uses 6.4 GB. > Partition 7 is Reiser4 -- uses 2.6 GB. > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > info). > > Don't you think this result is significant in itself? Only if you think disk space is so valuable that trading cpu time to compress and decompress the data is a good trade off. It is not one I would want to make. So you saved 3GB, what is that? About $1 worth? maybe $2 if you have raid. How much extra time and cpu will it take to access the data that way? How much extra electricity will the cpu use? What is your time worth? There are so many variables. Do you _trust_ reiserfs4 to not loose your data any more or less than some other filesystem? -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Fri, 06 Apr 2007 19:47:36 PDT, [EMAIL PROTECTED] said: > On Fri, 6 Apr 2007 11:21:19 -0400, "Jan Harkes" <[EMAIL PROTECTED]> > > With compression there is a pretty high probability that one corrupted > > byte or disk block will result in loss of a considerably larger amount > > of data. > > Bad blocks are NOT dealt with by the filesystem,... so your comment is > irrelevant, or just plain wrong. > > If your filesystem is writing to bad blocks, then throw away your > operating system. You know... occasionally, blocks go bad *after* you write to them. If you have an uncompressed filesystem, it's often possible to recover most of the file , and just have a few 512-byte blocks of zeros, simply by doing something like 'dd if=bad.file of=bad.file bs=512 conv=noerror' or careful applications of 'skip=N'. If it's compressed, you usually can't recover the rest of a compression group if a previous block is lost. (And for those who talk about backups - yes, taking backups is good. However, it's the rare laptop or desktop machine that can afford the luxury of RAID disks, and backups usually happen once a night, if that often. This means that if you've been working hard on something important all day, and the disk blows chunks at 4:30PM, you *will* be suddenly very concerned over exactly how much you can recover off the failing drive And yes, I'd *love* to have all my users connected to nice SAN systems that do snapshotting and remote replication to DR sites and all that - but have you ever *priced* a petabyte of SAN storage, the NAS gateways to serve it to users, and upgrading several tens of thousands of network ports to Gig-E? Hint - US$1M would get us through a pilot, and probably $5M and up to *start* deployment. Anybody wanna buy us an EMC DMX-3? :) http://www.emc.com/products/systems/symmetrix/DMX_series/DMX3.jsp pgplVLnYRAhwP.pgp Description: PGP signature
Re: Reiser4. BEST FILESYSTEM EVER.
On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said: > > Gzip - 3 files (zeros only, raw DV data from video camera, x86_64 > kernel rpm file), 10 MB of data (10*1024*1024), > $ l -Ggh zeros dv bin > -rw-r--r-- 1 10M Apr 7 15:30 bin > -rw-r--r-- 1 10M Apr 7 15:31 dv > -rw-r--r-- 1 10M Apr 7 15:31 zeros > $ l -Ggh zeros.gz dv.gz bin.gz > -rw-r--r-- 1 10K Apr 7 15:31 zeros.gz > -rw-r--r-- 1 9.1M Apr 7 15:31 dv.gz > -rw-r--r-- 1 9.3M Apr 7 15:30 bin.gz > > ... and though the numbers may still sound impressive, space savings > are less than 10%. I am quite sure that the kernel RPM file is *already* compressed, at least somewhat. Otherwise, it's hard to explain this: -rw-r--r--1 529 263 17835757 Apr 5 00:19 kernel-2.6.20-1.3045.fc7.x86_64.rpm % du -s /lib/modules/2.6.20-1.3038.fc7/ 76436 /lib/modules/2.6.20-1.3038.fc7/ and it can't all be slack space at ends of files: % find /lib/modules/2.6.20-1.3038.fc7/ -type f | wc -l 1482 Even on a 4K filesystem, the *max* wasted slack would be about 4M. And what do you know - if you tar.gz that /lib/modules: % tar czf /tmp/kern.tar.gz /lib/modules/2.6.20-1.3038.fc7/ tar: Removing leading `/' from member names % ls -l /tmp/kern.tar.gz -rw-r--r-- 1 valdis valdis 15506359 2007-04-07 13:19 /tmp/kern.tar.gz The *compressed* tar is about 15M (remember the .rpm contained a 2M vmlinuz as well - that;s compressed too). So we're right up to the 17M of the .rpm, which indicates that the RPM is compressed at a factor close to tar.gz. I'd not be surprised to find out that your digital-video also contains at least some light compression - if it's mpeg or similar, that's already had some *heavy* compression done to it pgpvTuIn4rkKi.pgp Description: PGP signature
Re: Reiser4. BEST FILESYSTEM EVER.
On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said: > > Think about it,... read speeds that are some FOUR times the physical > > disk read rate,... impossible without the use of compression (or > > something similar). > > It's really impossible with compression only unless you're writing > only zeros or stuff alike. I don't know what bonnie uses for testing > but real life data doesn't compress 4 times. Two times, sometimes, All depends on your data. From a recent "compress the old logs" job on our syslog server: /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308: 85.4% -- replaced with /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308.gz And it wasn't a tiny file either - it's a busy mailserver, the logs run to several hundred megabytes a day. Syslogs *often* compress 90% or more, meaning a 10X compression. > but then it will be typically slower than disk access (I mean read, > as write will be much slower). Actually, as far back as 1998 or so, I was able to document 20% *speedups* on an AIX system that supported compressed file systems - and that was from when a 133mz PowerPC 604e was a *fast* machine. Since then, CPUs have gotten faster at a faster rate than disks have, even increasing the speedup. The basic theory is that unless you're sitting close to 100%CPU, it is *faster* to burn some CPU to compress/decompress a 4K chunk of data down to 2K, and then move 2K to the disk drive, than it is to move 4K. It's particularly noticable for larger files - if you can apply the compression to remove the need to move 2M of data faster than you can move 2M of data, you win. pgp8M6yyVbuKG.pgp Description: PGP signature
Re: Reiser4. BEST FILESYSTEM EVER.
On 4/7/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I checked what bonnie++ actually writes to its test files, for you. It is about 98-99% zeros. Still, the results record sequential reads, of 232,729 K/sec, nearly four times the physical disk read rate, 63,160 K/sec, of the hard drive. Excellent! You've established the undeniable hard cold fact that reiser4 beats the crap out of all other filesystems, when the files are 98-99% filled with zeros. You've proven your point, so can we stop this thread now? - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Sat, 7 Apr 2007 13:59:14 +0100, "Dale Amon" <[EMAIL PROTECTED]> said: > Jan does have a point about bad blocks. A couple years ago > I had a relatively new disk start to go bad on random blocks. > I detected it fairly quickly but did have some data loss. > > All the compressed archives which were hit were near > total losses; most other files were at least partially > recoverable. As you know, there is not substitute for backups. What if the disk had totally crashed and scratched GBs of your data. And did you ever trust those (non-compressed) executables that you saved after recovering them from corruption? Of course not. No one would. The fact that they were not compressed did not save them. You are really arguing for backups, not for one filesystem or another. Besides, Jan claimed that corruption due to bad blocks propagates to MULTIPLE files because of the compression in the file system. You are arguing something different. > It is not a matter of your operating system writing > to bad blocks. It is a matter of what happens when the > blocks on which your data sit go bad underneath you. > > This issue has also been discussed by people working > with revision control system. If you are archiving > data, how do you know you if your data is still good > unless you actually need it? If you do not know it > is bad, you may well get rid of good copies thinking > you do not need the extras... it does happen. > > I would be quite hesitant to go with on disk compression > unless damage was limited to only the bad bits or blocks > and did not propagate through the rest of the file. You don't really mean that. Most backup uses compression (which propagates errors through the rest of the file). > Perhaps if everyone used hardware RAID and the RAID > automatically detected a difference due to trashed > data on one disk and flagged the admin with a warning... > > BTW: I'm a CMU Alum, so who are you working with John? I retired quite young. -- [EMAIL PROTECTED] -- http://www.fastmail.fm - Send your email first class - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Krzysztof -- Aren't you missing the point? Twice the speed would be great,... even a 50% increase,... even a 0% increase. I checked what bonnie++ actually writes to its test files, for you. It is about 98-99% zeros. Still, the results record sequential reads, of 232,729 K/sec, nearly four times the physical disk read rate, 63,160 K/sec, of the hard drive. The sequential writes are about three times the physical disk write rate. Even if the speed increase was zero, the more efficient use of disk space means that Reiser4 is worth investigating. People use RAID arrays to achieve speed increases. The people who developed RAID clearly thought that increases in speed were worth investigating. > > > Why do you think I hate reiserfs developers? That is an insane claim. > > Why would I hate reiser3 developers? > > Why would I hate reiser4 developers? > > Why would I even dislike them? > > > > I think Hans Reiser is a genius. Is that what you mean by hate? > > I think they could hire a person with a bit better marketing skills, > though. People on a technical mailing list don't buy things just > because something on TV told them they have to. I don't work for Reiser if that is what you are suggesting. And people buy all sorts of lies because someone on TV told them it was true. Did you believe Iraq had WMD (weapons of mass destruction) because a bunch of American liars told you this on TV? Millions of Americans did. > > Answer this question. Why do YOU think I am antagonizing reiserfs > > developers? > > That might be just a side effect. > -- [EMAIL PROTECTED] -- http://www.fastmail.fm - Send your email first class - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
[EMAIL PROTECTED] writes: > Why do you think I hate reiserfs developers? That is an insane claim. > Why would I hate reiser3 developers? > Why would I hate reiser4 developers? > Why would I even dislike them? > > I think Hans Reiser is a genius. Is that what you mean by hate? I think they could hire a person with a bit better marketing skills, though. People on a technical mailing list don't buy things just because something on TV told them they have to. > Answer this question. Why do YOU think I am antagonizing reiserfs > developers? That might be just a side effect. > Think about it,... read speeds that are some FOUR times the physical > disk read rate,... impossible without the use of compression (or > something similar). It's really impossible with compression only unless you're writing only zeros or stuff alike. I don't know what bonnie uses for testing but real life data doesn't compress 4 times. Two times, sometimes, but then it will be typically slower than disk access (I mean read, as write will be much slower). You can get faster I/O (both linear speed and access times) using multiple disks (mirrors etc). Perhaps some ZFS ideas would do us some good? Gzip - 3 files (zeros only, raw DV data from video camera, x86_64 kernel rpm file), 10 MB of data (10*1024*1024), done on tmpfs so no real disk speed factor. The CPU is AMD64 with 1 MB cache per core, 2600 MHz clock (clock scaling disabled). That's my typical usage pattern (well, not counting these zeros). $ l -Ggh zeros dv bin -rw-r--r-- 1 10M Apr 7 15:30 bin -rw-r--r-- 1 10M Apr 7 15:31 dv -rw-r--r-- 1 10M Apr 7 15:31 zeros $ for f in zeros dv bin; do time gzip $f; done real0m0.112s real0m0.686s real0m0.559s Dealing with pure zeros gzip can get almost 90 MB/s compressing, but with DV and rpm it only does 14.5 and almost 18 MB/s respectively... $ l -Ggh zeros.gz dv.gz bin.gz -rw-r--r-- 1 10K Apr 7 15:31 zeros.gz -rw-r--r-- 1 9.1M Apr 7 15:31 dv.gz -rw-r--r-- 1 9.3M Apr 7 15:30 bin.gz ... and though the numbers may still sound impressive, space savings are less than 10%. $ for f in zeros dv bin; do time gunzip $f.gz; done real0m0.067s real0m0.131s real0m0.120s Decompression gives 150 MB/s for zeros and ~ 80 MB/s for DV and rpm. $ for f in zeros dv bin; do time gzip -1 $f; done real0m0.079s real0m0.572s real0m0.530s Supposed to be "fastest gzip". 126 MB/s for zeros but still less than 19 MB/s for DV and rpm. $ l -Ggh zeros.gz dv.gz bin.gz -rw-r--r-- 1 45K Apr 7 15:31 zeros.gz -rw-r--r-- 1 9.2M Apr 7 15:31 dv.gz -rw-r--r-- 1 9.3M Apr 7 15:30 bin.gz $ for f in zeros dv bin; do time gunzip $f.gz; done real0m0.044s real0m0.135s real0m0.120s It seems gzip can decompress zeros with 227 MB/s rate. I assume the "4x read speed" claim comes from something like this. $ /sbin/hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 210 MB in 3.02 seconds = 69.59 MB/sec $ echo "69.59*4" | bc 278.36 Seems you'd need a faster algorithm, faster machine or slower disk - slower than this cheap SATA with disabled NCQ (NV SATA) at least: $ cat /sys/block/sda/device/model Maxtor 6V250F0 Please note that aplication-level compression usually gives way better results - the application knows much more. -- Krzysztof Halasa - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Jan does have a point about bad blocks. A couple years ago I had a relatively new disk start to go bad on random blocks. I detected it fairly quickly but did have some data loss. All the compressed archives which were hit were near total losses; most other files were at least partially recoverable. It is not a matter of your operating system writing to bad blocks. It is a matter of what happens when the blocks on which your data sit go bad underneath you. This issue has also been discussed by people working with revision control system. If you are archiving data, how do you know you if your data is still good unless you actually need it? If you do not know it is bad, you may well get rid of good copies thinking you do not need the extras... it does happen. I would be quite hesitant to go with on disk compression unless damage was limited to only the bad bits or blocks and did not propagate through the rest of the file. Perhaps if everyone used hardware RAID and the RAID automatically detected a difference due to trashed data on one disk and flagged the admin with a warning... BTW: I'm a CMU Alum, so who are you working with Jan? - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Hi Willy,... > With decent CPU, you can reach higher read/write data rates than what a > single off-the-shelf disk can achieve. For this reason, I think that > reiser4 would be worth trying for this particular usage. Glad to see you are willing to give Reiser4 a go. Good man. -- On Sat, 7 Apr 2007 09:15:35 +0200, "Willy Tarreau" <[EMAIL PROTECTED]> said: > On Fri, Apr 06, 2007 at 10:58:45PM -0700, [EMAIL PROTECTED] > wrote: > > You know,... you cut out this bit: > > > > - > > > > > The following benchmarks are from > > > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > ... > > Hey John, please change your disk, it's scratched and you're repeating > yourself again and again. At first I thought "Oh cool, some good news > about reiser4", now when I see "reiserfs" in a thread, I think "oh no, > not this boring guy who escaped from the asylum again !". I hope this > thread will be cut shortly so that you stop doing bad publicity to > reiserfs and its developers, because when a product is indicated as > good by stupid people, it's really doing harm. > > Also, about this part : > [Jan] > > > But in the end everything is a tradeoff. You can save diskspace, but > > > increase the cost of corruption. > > I don't 100% agree with Jan, because for some usages (temporary space), > light compression can increase speed. For instance, when processing logs, > I get better speed by compressing intermediate files with LZO on the fly. > > [John] > > You deliberately ignored the fact that bad blocks are NOT dealt with by > > the filesystem,... but by the operating system. Like I said: If your > > filesystem is writing to bad blocks, then throw away your operating > > system. > > But what you write here is complete crap. The filesystem relies on a > linear block device. The operating system is responsible for doing > read retries or reporting errors on bad blocks, but the FS and only > the FS can decide how not to use some known defective areas, for > instance not putting any metadata on them nor any useful data. > > Now if you want to stop writing stupid things again and again, take > your bag, don't miss the bus to school, and listen to the teachers > instead of playing games on your calculator. > > Willy > PS: non need to reply either, I'll kill this thread and your address > here. > -- [EMAIL PROTECTED] -- http://www.fastmail.fm - One of many happy users: http://www.fastmail.fm/docs/quotes.html - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Fri, Apr 06, 2007 at 10:58:45PM -0700, [EMAIL PROTECTED] wrote: > You know,... you cut out this bit: > > - > > > The following benchmarks are from > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm ... Hey John, please change your disk, it's scratched and you're repeating yourself again and again. At first I thought "Oh cool, some good news about reiser4", now when I see "reiserfs" in a thread, I think "oh no, not this boring guy who escaped from the asylum again !". I hope this thread will be cut shortly so that you stop doing bad publicity to reiserfs and its developers, because when a product is indicated as good by stupid people, it's really doing harm. Also, about this part : [Jan] > > But in the end everything is a tradeoff. You can save diskspace, but > > increase the cost of corruption. I don't 100% agree with Jan, because for some usages (temporary space), light compression can increase speed. For instance, when processing logs, I get better speed by compressing intermediate files with LZO on the fly. [John] > You deliberately ignored the fact that bad blocks are NOT dealt with by > the filesystem,... but by the operating system. Like I said: If your > filesystem is writing to bad blocks, then throw away your operating > system. But what you write here is complete crap. The filesystem relies on a linear block device. The operating system is responsible for doing read retries or reporting errors on bad blocks, but the FS and only the FS can decide how not to use some known defective areas, for instance not putting any metadata on them nor any useful data. Now if you want to stop writing stupid things again and again, take your bag, don't miss the bus to school, and listen to the teachers instead of playing games on your calculator. Willy PS: non need to reply either, I'll kill this thread and your address here. - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
On Fri, 6 Apr 2007 23:30:49 -0400, "Jan Harkes" <[EMAIL PROTECTED]> said: > > Since you decide to publically respond to a private email, but not only > you did not 'discuss' anything I wrote and in fact cut out most of the > useful information in my reply I guess I will have to repeat my > observations. You are a funny guy Jan. Here you are, once again, cutting out my most useful information, ie, the data I was discussing, while complaining that I cut out your most useful information. You know,... you cut out this bit: - > The following benchmarks are from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > .-. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2| 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4| 4408 | 816 | > |EXT3| 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-. > > Column one measures the time taken to complete the bonnie++ benchmarking test > (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB of > raw data (which was 3 different copies of the Linux kernel sources). - And this bit: Jan,... Here is another section, you conveniently cut out. Maybe you should explain why you cut out this section? Was it embarrassing to you? I mean, your statement is sort of correct,... but it shows a basic misunderstanding of filesystems. > > With compression there is a pretty high probability that one corrupted > > byte or disk block will result in loss of a considerably larger amount > > of data. > > Bad blocks are NOT dealt with by the filesystem,... so your comment is > irrelevant, or just plain wrong. > > If your filesystem is writing to bad blocks, then throw away your > operating system. - It is true that I considered your "most useful information," an irrelevant section, which is why it was cut out (ignored). I did not see my doing this, any worse than you doing it. I did not realize that you were be so impolite. As to your email being private, I had thought I had joined a mailing list. I had not idea your email was meant to be private and just considered it like all the others. Now you mention it, I wondered why the email did not automatically list the mailing lists, as recipients, and I had to add them. If I had realized this I may have added the mailing lists as recipients, anyway. It would be like me to do such. However, you should understand that I am new to mailing lists. > > > > Do you really have to repeat the results in every email you sent? > > > > Damn, I did it again. WHY DO YOU CARE? > > Because I saw them the first time around. And although the performance > difference on those micro benchmarks seems quite impressive, I'm not > convinced. So, likewise, I saw your comments (you know the ones you miss so much) the first time around, as I was not convinced of their worth. The benchmarks measure certain data. Its fine you do not read into them, stuff that isn't there, like reliability, for example. > > Look, its simple, I am (among other things) discussing these results, so > > people need to see them. > > However, you do not discuss, you just repeat, and repeat, and repeat. I never said I wanted discussion, you just said I did. You just repeat, and repeat, and repeat. In reality, I quite appreciate reasonable discussion. But, I doubt I will get much from you. > But for what reason. Do you want an actual discussion, or do you hate > the reiserfs developers so much that you want to antagonize any and all > other Linux file systems developers? Why do you think I hate reiserfs developers? That is an insane claim. Why would I hate reiser3 developers? Why would I hate reiser4 developers? Why would I even dislike them? I think Hans Reiser is a genius. Is that what you mean by hate? Answer this question. Why do YOU think I am antagonizing reiserfs developers? You must have a reason for stating what you have. > > By the way, I have pulled the plug on my REISER4 system, a number of > > times now, and it recovers without problem. > > Very nice for you. I guess you have also not been hit by out of memory > conditions or failing partial writes. So I guess we can ignore the patch > that was just sent a day or two ago to the mailing list because you > succesfully pulled the plug, a number of times at that. Why are you attacking me with sarcasm, when I have just stated a simple fact? > > > > I have set up a Reiser4 partitio
Re: Reiser4. BEST FILESYSTEM EVER.
Since you decide to publically respond to a private email, but not only you did not 'discuss' anything I wrote and in fact cut out most of the useful information in my reply I guess I will have to repeat my observations. Once I send out this email, I'll just add you to my friendly killfile (as well as this thread's subject/msgids) so don't bother continuing you one-sided 'discussions' on this topic. On Fri, Apr 06, 2007 at 07:47:36PM -0700, [EMAIL PROTECTED] wrote: > > Do you really have to repeat the results in every email you sent? > > Damn, I did it again. WHY DO YOU CARE? Because I saw them the first time around. And although the performance difference on those micro benchmarks seems quite impressive, I'm not convinced. > Look, its simple, I am (among other things) discussing these results, so > people need to see them. However, you do not discuss, you just repeat, and repeat, and repeat. But for what reason. Do you want an actual discussion, or do you hate the reiserfs developers so much that you want to antagonize any and all other Linux file systems developers? > > > Don't you agree, that "If they are accurate, THEN they are obviously > > > very relevant." > > > > Not everyone does. I care mostly about reliability and availability > > neither of which are shown by your results. > > Actually, to some extent, bonnie++ tests the reliability of the > filesystem, eg, NTFS-3g usually fails. > > By the way, I have pulled the plug on my REISER4 system, a number of > times now, and it recovers without problem. Very nice for you. I guess you have also not been hit by out of memory conditions or failing partial writes. So I guess we can ignore the patch that was just sent a day or two ago to the mailing list because you succesfully pulled the plug, a number of times at that. > > > I have set up a Reiser4 partition with gzip compression, here is the > > > difference in disk usage of a typical Debian installation on two 10GB > > > partitions, one with Reiser3 and the other with Reiser4. > > > > > > debian:/# df > > > Filesystem 1K-blocks Used Available Use% Mounted on > > > /dev/sda3 10490104 6379164 4110940 61% /3 > > > /dev/sda7 9967960 2632488 7335472 27% /7 > > ... > > > Partition 3 is Reiser3 -- uses 6.4 GB. > > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > > info). > > > > Wow, consider me totally and completely, unimpressed. > > Here is the part of my email that you seemed to totally ignore, You've just saved yourself $3.80, now go get yourself a latte. (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) Seriously, disk storage is getting less expensive all the time, you can already buy a 250GB SATA drive for $70. Also, compression doesn't help once you store already compressed data such as jpeg images, mp3 files, or mpeg2/4/divx video files. Not only are the savings non-existant, but we still end up with the disadvantage that corruption propagates to multiple files because of the compression in the file system. And if it doesn't propagate across multiple files, the compression can't be all that good since it can't benefit as much from similarity between files. So if that is the case and you really want to save diskspace you almost have to look at read-only compressed filesystems such as cramfs, squashfs, zisofs, cloop and various other variants in combination with a unionfs overlay to get read/write functionality. But in the end everything is a tradeoff. You can save diskspace, but increase the cost of corruption. Use a better compression algorithm, but that uses more CPU or which is visible in performance of the application. This can be offset by caching more, and being lazier with writebacks, but that hurts on-disk consistency, creates more memory pressure (more swapout/paging) and generally slows down other applications that aren't actually accessing the disk. Having a fast multi-core cpu and lots of memory helps a lot, but at some point what tradeoff did we just make, we saved a couple of dollars not having to buy a larger disk, but we spend considerably more on the more expensive cpu and memory. > Wow, consider me totally impressed by your AMAZING BIAS. > > Would you like to tell me why you are SO BIASED against REISER4. And that is the reponse I get, I thought you wanted discussion, but clearly you don't care about any meaningful discussion. Your goal seems to be to make sure that other developers end up ignoring any thread that has reiser in the subject. And even if they are not biased and welcome a discussion, you will just call them out on it, because clearly if they want to discuss something they aren't totally with it, so they have to be totally
Re: Reiser4. BEST FILESYSTEM EVER.
On Fri, 6 Apr 2007 11:21:19 -0400, "Jan Harkes" <[EMAIL PROTECTED]> said: > Do you really have to repeat the results in every email you sent? The following benchmarks are from http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm .-. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2| 4092 | 816 | |JFS | 4225 | 806 | |EXT4| 4408 | 816 | |EXT3| 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). > Do you really have to repeat the results in every email you sent? Damn, I did it again. WHY DO YOU CARE? Look, its simple, I am (among other things) discussing these results, so people need to see them. > > Don't you agree, that "If they are accurate, THEN they are obviously > > very relevant." > > Not everyone does. I care mostly about reliability and availability > neither of which are shown by your results. Actually, to some extent, bonnie++ tests the reliability of the filesystem, eg, NTFS-3g usually fails. By the way, I have pulled the plug on my REISER4 system, a number of times now, and it recovers without problem. > With compression there is a pretty high probability that one corrupted > byte or disk block will result in loss of a considerably larger amount > of data. Bad blocks are NOT dealt with by the filesystem,... so your comment is irrelevant, or just plain wrong. If your filesystem is writing to bad blocks, then throw away your operating system. > > I have set up a Reiser4 partition with gzip compression, here is the > > difference in disk usage of a typical Debian installation on two 10GB > > partitions, one with Reiser3 and the other with Reiser4. > > > > debian:/# df > > Filesystem 1K-blocks Used Available Use% Mounted on > > /dev/sda3 10490104 6379164 4110940 61% /3 > > /dev/sda7 9967960 2632488 7335472 27% /7 > ... > > Partition 3 is Reiser3 -- uses 6.4 GB. > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > info). > > Wow, consider me totally and completely, unimpressed. > Wow, consider me totally impressed by your AMAZING BIAS. Would you like to tell me why you are SO BIASED against REISER4. John. -- [EMAIL PROTECTED] -- http://www.fastmail.fm - A fast, anti-spam email service. - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Reiser4. BEST FILESYSTEM EVER.
Hi Peter, You say that the results may be accurate, but "Whether or not they're *relevant* is a totally different ball of wax." and "Whether or not they're relevant depends on how well they happen to reflect your particular usage pattern." Well, surprise, surprise,.. everyone knows that. Have a look at the (summary) of the results: .-. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2| 4092 | 816 | |JFS | 4225 | 806 | |EXT4| 4408 | 816 | |EXT3| 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-. for the full results see: http://linuxhelp.150m.com/resources/fs-benchmarks.htm Don't you agree, that "If they are accurate, THEN they are obviously very relevant." I have set up a Reiser4 partition with gzip compression, here is the difference in disk usage of a typical Debian installation on two 10GB partitions, one with Reiser3 and the other with Reiser4. debian:/# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 10490104 6379164 4110940 61% /3 /dev/sda7 9967960 2632488 7335472 27% /7 Partitions 3 and 7 have exactly the same data on them (the typical Debian install). The partitions are exactly the same size (although df records different sizes). Partition 3 is Reiser3 -- uses 6.4 GB. Partition 7 is Reiser4 -- uses 2.6 GB. So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same info). Don't you think this result is significant in itself? Following your hint I have booted /dev/sda7 and all the programs seem to work fine. They do not seem to be any faster than when using Reiser3. The whole system seems about as responsive as always. For fun, I ran bonnie++. Here are the results: debian:/# ./bonnie++ -u root Using uid:0, gid:0. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c --Sequential Output-- --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP debian 1G 121 99 86524 21 63297 41 920 99 187762 80 1782 233 Latency 82484us 386ms 438ms 26758us 110ms 398ms Version 1.93c --Sequential Create-- Random Create debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ 18509 92 17776 86 + +++ 19495 91 Latency 210us5475us5525us5777us5522us 5839us I particularly liked the 233%CP for Random-Seeks. John. On Thu, 05 Apr 2007 21:07:28 -0700, "H. Peter Anvin" <[EMAIL PROTECTED]> said: > [EMAIL PROTECTED] wrote: > > Hi Peter, > > > > You say that the results may be accurate, but not relevant. > > > > NO, I said that whether they're accurate is another matter. > > > If they are accurate, THEN they are obviously very relevant. > > Crap-o-la. Whether or not they're relevant depends on how well they > happen to reflect your particular usage pattern. > > There are NO benchmarks which are relevant to all users. Understanding > whether or not a benchmark is relevant to one's particular application > is one of the trickiest things about benchmarks. > > -hpa -- [EMAIL PROTECTED] -- http://www.fastmail.fm - Email service worth paying for. Try it for free - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
