Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Wed, 28 Jul 2010 21:34:06 +0200 Aniruddha mailingdotl...@gmail.com wrote: On Wed, Jul 28, 2010 at 9:09 PM, Perry E. Metzger pe...@piermont.com wrote: On Wed, 28 Jul 2010 14:46:29 +0200 Aniruddha mailingdotl...@gmail.com wrote: I have done some testing with Debian stable in Virtualbox and I have to say XFS works as advertised. I did power off the virtual machine several times when working in Gnome / copying files. And I did power off the virtual 5 times in a row when booting. Nothing happened. Each time the virtual machine booted without problems. I have to say, file system creating and file system checking is lightning fast. I am very impressed. Next I'll test XFS on my laptop. Although I have no reason to believe that XFS is flawed, your test is not proof of that. Agreed, it was hardly a double-blind randomized trial :) You make it sound like it was somehow useful if not entirely rigorous. In fact, the exercise showed virtually nothing at all. On a more serious note: off course these tests don't prove anything. On the other hand I have heard so many time that XFS can't handle a single power failure without data corruption that I wanted to see for myself what happens if you power off a pc. And you haven't learned that even now, because virtual hardware does not behave enough like real hardware. For example, the virtual hardware does not simulate a modern disk cache at all, let alone the behavior of such a cache on a true power cut, so you would not see problems associated with the disk cache silently reordering writes and then failing to complete all of them on power failure. I would suggest avoiding making any pronouncements based on such experiments. I have no reason to believe XFS has any problems at all, but your test did not demonstrate anything of value. Although most experienced users would dismiss your information as worthless, you might convince some people with insufficient knowledge that you had actually determined something and that they may make a decision on the basis of your experiments. -- Perry E. Metzgerpe...@piermont.com -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100729100305.122a8...@jabberwock.cb.piermont.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Dne, 28. 07. 2010 03:53:43 je Paul E Condon napisal(a): Stan, Have you ever heard of the term 'invincible ignorance'? Also, you have asserted with some vigor upstream Your original post ... IMHO, was correct but somewhat harshly worded. Peace. IMHO, Stan's deep, invaluable expertise is sometimes, regrettably, undermined by his own vigor and harshly worded attitudes; some people may be put off by his dismissive tone. I know I am. As is Volkan Yazici, if we may judge from one of his replies to Stan: BTW, I still couldn't understand your temper and rudeness. -- Peace. Klistvud, one of the unwashed mass of invincibly ignorant people Certifiable Loonix User #481801 http://bufferoverflow.tiddlyspot.com -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1280307691.2684...@compax
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
I have done some testing with Debian stable in Virtualbox and I have to say XFS works as advertised. I did power off the virtual machine several times when working in Gnome / copying files. And I did power off the virtual 5 times in a row when booting. Nothing happened. Each time the virtual machine booted without problems. I have to say, file system creating and file system checking is lightning fast. I am very impressed. Next I'll test XFS on my laptop. dmesg --- [5.182230] sda: sda1 sda2 [5.183801] sd 0:0:0:0: [sda] Attached SCSI disk [5.324028] PM: Starting manual resume from disk [5.77] SGI XFS with ACLs, security attributes, realtime, large block numbers, no debug enabled [5.334413] SGI XFS Quota Management subsystem [5.335916] XFS mounting filesystem sda2 [5.420162] Starting XFS recovery on filesystem: sda2 (logdev: internal) [5.439531] Ending XFS recovery on filesystem: sda2 (logdev: internal) [6.796197] udevd version 125 started -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktinwin-pg10egjljgva4imaacgwoca5gnecyf...@mail.gmail.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Paul E Condon put forth on 7/27/2010 8:53 PM: Your original post in this thread addressed a quite disfunctional attitude of OP, and IMHO, was correct but somewhat harshly worded. In truth, he simply cannot have everything he wants all at the same time. You should have left it at that, IMHO. Words of wisdom, you speak. Retired from this thread, I have. Let go of harsh wording, I shall. :) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c506128.3020...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Wed, 28 Jul 2010 14:46:29 +0200 Aniruddha mailingdotl...@gmail.com wrote: I have done some testing with Debian stable in Virtualbox and I have to say XFS works as advertised. I did power off the virtual machine several times when working in Gnome / copying files. And I did power off the virtual 5 times in a row when booting. Nothing happened. Each time the virtual machine booted without problems. I have to say, file system creating and file system checking is lightning fast. I am very impressed. Next I'll test XFS on my laptop. Although I have no reason to believe that XFS is flawed, your test is not proof of that. Most journaling and similar bugs are found only in rare conditions. You would need to conduct your test hundreds of thousands of times, under a variety of conditions, including differing hardware cache reorder policies, differing file system loads, differing numbers of concurrent processes within the kernel code paths, etc. Real torture tests done by people trying to nail down bugs often involve millions of iterations in randomized test harnesses. Five tests is hardly enough to find even common bugs. If a test like the one you conducted were meaningful, the jobs of systems programmers would be much simpler. We could just try something a couple of times by hand and we would know if our code was flawless. Sadly, the world does not work that way. -- Perry E. Metzgerpe...@piermont.com -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100728150947.2dd7d...@jabberwock.cb.piermont.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Wed, Jul 28, 2010 at 9:09 PM, Perry E. Metzger pe...@piermont.com wrote: On Wed, 28 Jul 2010 14:46:29 +0200 Aniruddha mailingdotl...@gmail.com wrote: I have done some testing with Debian stable in Virtualbox and I have to say XFS works as advertised. I did power off the virtual machine several times when working in Gnome / copying files. And I did power off the virtual 5 times in a row when booting. Nothing happened. Each time the virtual machine booted without problems. I have to say, file system creating and file system checking is lightning fast. I am very impressed. Next I'll test XFS on my laptop. Although I have no reason to believe that XFS is flawed, your test is not proof of that. Agreed, it was hardly a double-blind randomized trial :) On a more serious note: off course these tests don't prove anything. On the other hand I have heard so many time that XFS can't handle a single power failure without data corruption that I wanted to see for myself what happens if you power off a pc. with an XFS filesystem. Apparently not much. There might other problems hidden with XFS,just like ext3 ( when copy pasting a home directory to another location I once lost the whole directory due data corruption on ext3). -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktikgp3px0huxn4ntbwwtxtsgemnmfdemkcm=0...@mail.gmail.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Op 28-07-10 21:34, Aniruddha schreef: Agreed, it was hardly a double-blind randomized trial :) On a more serious note: off course these tests don't prove anything. On the other hand I have heard so many time that XFS can't handle a single power failure without data corruption that I wanted to see for myself what happens if you power off a pc. with an XFS filesystem. Apparently not much. There might other problems hidden with XFS,just like ext3 ( when copy pasting a home directory to another location I once lost the whole directory due data corruption on ext3). As with all research, showing that something does not fail is notoriously hard. Showing that something fails given a specified set of circumstances, on the other hand, is dead easy. Yes, you did show that a poweroff does not destroy data. By the way, were you writing data? I think there's no filesystem that *can* fail when data is only being read, sort of hard disk failure of course... Sjoerd signature.asc Description: OpenPGP digital signature
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tuesday 27 July 2010 15:09:08 Stan Hoeppner wrote: Volkan YAZICI put forth on 7/27/2010 2:04 PM: unplugged machine. At boot, I dropped to fsck command line. At command Were you forced to the command line or did you manually select to go to the command line? It sounds like you chose to, not forced to. prompt, I manually fiddled around with fsck of xfs to recover the unmounted / filesystem, but had no luck. Did you read the xfs documentation before embarking on this power loss experiment? Or did you it should just work regardless of your actions, or lack of action? It sounds like you ran xfs_repair on a filesystem in an inconsistent state and forced changes, which is a no-no. (I also tried recommendations and informative messages supplied by manpages and command outputs/warnings.) Also if you would Google, it shouldn't be hard to spot similar experiences from other people. I'm guessing most of them didn't look before taking the XFS leap. At NASA, they might have genius technicians; but, IMHO a majority of the linux users would want a filesystem to recover without a prompt from the user. So the system wouldn't boot and you were dropped to a prompt. You manually fiddled around with fsck of xfs and made no progress. It would be nice to have seen all of that at the time. What were your results when you did this same power yank test with ext2/3, ReiserFS, and the other filesystems you tested in this way? I'm basically a one man army trying to defeat misinformation WRT XFS and attempt to educate ppl with the correct information. I am glad -users ml have you; and I'd be really, really appreaciated if somebody having experience and knowledge on fs issues can shed some light to our ignorance. I also support the replacement of default fs with something that is much more recent. From this point of view, XFS is a superior alternative. You are totally right with your claims about its advantages over other alternatives. But as you can see, people still complain about XFS's sensitivity to power failures. Assuming a majority of your users aren't behind a UPS, you can sell/ship your product with such a default filesystem choice. But as you said, there are no published concrete benchmarks about this issue. It is all what people claim in the mailing lists. If you would share some of your findings about Power Failures and XFS to convince us, I'm sure most of us will be happy to advocate XFS's this achievement. I've tried to dig up accurate accounts of the power loss corruption issue post 2007 (when it was supposed to have been fixed) a few times but couldn't find anything concrete enough to be worth referencing. I freely admit I've done no power loss testing of XFS myself. This probably has to do with the fact that I'm a firm believer in orderly shutdowns and redundant power, and that I don't really have any test systems available. I'll ask around on the XFS list and see what folks have to say. I'm somewhat interested in seeing where BTRFS is in 2-3 years. It may be stable enough for production by then, and should be as fast or faster than XFS on some workloads. Maybe it'll even handle sudden power loss gracefully. :) I've been using btrfs for my / file system for a few months now on my laptop. Toward the beginning I did suffer some dpkg database corruption due to a dpkg bug[1] which is now fixed in testing. It has handled sudden power failure, kernel hangs, suspend/resume to disk/memory, all gracefully. I am a bit concerned that there is no fsck.btrfs that will run at boot time, yet. However, that is less of an issue than it would be with other file systems since I can do online fsck and defragmentation. (Reminder to self: Write cron jobs.) I still would not advocate it in a critical production environment. My tests haven't been conclusive. The various user-space tools feel immateure and poorly documented. If it continues to improve, and does not hit any significant roadblocks, it may be ready for production around the Squeeze+1 release. My production systems still use ext3 file systems, but that is because they are simply VPS slices, so their initial installation is done automatically by my provider. Were I do be doing the initial installation myself, I would likely choose XFS. [1] dpkg made some assumptions about file system behavior when traversing a directory and renaming files in that directory. Those assumptions were not guaranteed by the POSIX / SUS specifications or the LSB. Btrfs violates those assumptions. So, while the bug was in dpkg, only using dpkg on top of Btrfs showed the bug. -- Boyd Stephen Smith Jr. ,= ,-_-. =. b...@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/\_/ signature.asc Description: This is a digitally signed message part.
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Lisi put forth on 7/27/2010 2:23 AM: On Tuesday 27 July 2010 08:10:15 Stan Hoeppner wrote: XFS which is superior to all other Linux filesystems. Stan - Have you the time to give a rationale for this? Sure. 1. Best overall performance for most systems, large and small, and the FS creation and mounting parameters are super configurable to match the system hardware for best performance. One recent set of recent benchmarks demonstrating so: http://btrfs.boxacle.net/repository/raid/2010-04-14_2004/2.6.34-rc3/2.6.34-rc3.html man mkfs.xfs man mount Older benchmarks: http://www.debian-administration.org/articles/388 http://everything2.com/index.pl?node_id=1479435 In regard to this last benchmark, some(many?) of the default XFS filesystem creation parameters and mounting parameters have changed. Note the testing was performed in 2005. A lot changes in 5 years. Read all you can and ask questions on the XFS mailing list before tweaking parameters based on what you find in old forum posts and benchmarks such as this. Guaranteed Rate I/O for streaming and other critical applications--unique to XFS amongst all filesytems, ever, not just on Linux--this feature was born on IRIX XFS for the broadcasting industry where video stutter was basically death to a TV station or network such as CNN, CBS, etc. This single feature from SGI allowed broadcast media to wholesale convert from tape to disk (this and SGI FC storage arrays) 2. Commercial origin and backing. SGI is a fantastic technology compay: http://oss.sgi.com/projects/xfs/ 3. Maturity/history/longevity, IRIX birth in 1993, Linux birth 2001, included in mainline in late 2003: http://en.wikipedia.org/wiki/XFS http://marc.info/?l=linux-kernelm=107088371607817w=2 4. Equal/superior user space toolset: xfsprogs - includes online defragmentation tool xfs_fsr and online growth tool xfs_growfs. No other stable Linux FS has an online defragmenter. Ext4 has e4defrag but AFAIK it's not complete nor close to maturity or stability. xfs_fsr has been both for a decade. 5. Very active developer community and thorough documentation: http://xfs.org/index.php/Main_Page I'm not in any way impugning your knowledge. But I am at the stage of accepting the default that Lenny gives me, for no better reason than that the developers chose it and it is there. It is time I understood better the reasons for each file system. (I gave up Reiserfs because Reiser murdered his wife - hardly a logical measure of how good his filesystem is!!) Debian will _always_ default to an EXT* filesystem--until the end of time. Then again, I thought the same of LILO, so what do I know eh? But expert install mode allows you whatever you want. I never cared for ReiserFS and never used it. Hans actions are just further justification after the fact. I've only used XFS on servers. I've never used it on laptops or desktops. I know of many people who have, but they are die hard propeller heads and know how to fix anything if/when it breaks. If you want to run XFS on a laptop or desktop, especially on Debian, your only downside is going to be getting quick help, if you get jammed, from folks on this list, or the community in general. That process will probably not be as quick and fruitful as with EXT2/3 issues, simply because there are a lot less people using XFS, thus the pool of helpers is much smaller. If you're adventurous, take XFS for a spin. Partition a 100MB /boot and install everything else on a big partition running XFS. And, learn the utility set, and learn about XFS, just as you have (or should have) with EXT2/3 and ReiserFS. As always, knowledge is power. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4e9e29.4050...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 01:51, Stan Hoeppner s...@hardwarefreak.com wrote: Debian will _always_ default to an EXT* filesystem--until the end of time. Nope, btrfs will replace ext3/4 as default soon enough. Cheers, Kelly Clowers -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlkti=hgrom682lbgmypx44uqj2dg25jwxzhivva...@mail.gmail.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: 1. Best overall performance for most systems, large and small, and the FS creation and mounting parameters are super configurable to match the system hardware for best performance. One recent set of recent benchmarks demonstrating so: http://btrfs.boxacle.net/repository/raid/2010-04-14_2004/2.6.34-rc3/2.6.34-rc3.html man mkfs.xfs man mount Older benchmarks: http://www.debian-administration.org/articles/388 http://everything2.com/index.pl?node_id=1479435 In regard to this last benchmark, some(many?) of the default XFS filesystem creation parameters and mounting parameters have changed. Note the testing was performed in 2005. A lot changes in 5 years. Read all you can and ask questions on the XFS mailing list before tweaking parameters based on what you find in old forum posts and benchmarks such as this. Guaranteed Rate I/O for streaming and other critical applications--unique to XFS amongst all filesytems, ever, not just on Linux--this feature was born on IRIX XFS for the broadcasting industry where video stutter was basically death to a TV station or network such as CNN, CBS, etc. This single feature from SGI allowed broadcast media to wholesale convert from tape to disk (this and SGI FC storage arrays) 2. Commercial origin and backing. SGI is a fantastic technology compay: http://oss.sgi.com/projects/xfs/ 3. Maturity/history/longevity, IRIX birth in 1993, Linux birth 2001, included in mainline in late 2003: http://en.wikipedia.org/wiki/XFS http://marc.info/?l=linux-kernelm=107088371607817w=2 4. Equal/superior user space toolset: xfsprogs - includes online defragmentation tool xfs_fsr and online growth tool xfs_growfs. No other stable Linux FS has an online defragmenter. Ext4 has e4defrag but AFAIK it's not complete nor close to maturity or stability. xfs_fsr has been both for a decade. 5. Very active developer community and thorough documentation: http://xfs.org/index.php/Main_Page You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. And considering my personal experiences, reiserfs is the fastest fs (among ext3 and xfs) in terms of boot recovery phase times. Regards. [1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html [2] http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87bp9tuki8@alamut.alborz.net
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 3:22 PM, Volkan YAZICI yazic...@ttmail.com wrote: You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. Ext3 has the same problems when not properly configured: Ext3 does not do checksumming when writing to the journal. If barrier=1 is not enabled as a mount option (in /etc/fstab), and if the hardware is doing out-of-order write caching, one runs the risk of severe filesystem corruption during a crash. http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal For the record I use ext3, I remember XFS as not being reliable enough (with power failures etc).
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On 7/27/2010 1:23 AM, Lisi wrote: On Tuesday 27 July 2010 08:10:15 Stan Hoeppner wrote: XFS which is superior to all other Linux filesystems. Stan - Have you the time to give a rationale for this? Except XFS filesystems can't shrink, only grow. Sucks when you need to resize partitions/volumes, and they're all XFS. Further, XFS makes more system calls to the kernel than standard Ext2/3/4. Export an XFS filesystem on LVM over NFS, and you'll get a kernel oops on a 32-bit kernel. Trace it, and you'll see the plethora of nested system calls XFS makes. You won't oops with Ext2/3/4 in the same scenario. This can be mitigated by running a 64-bit system, if you have the hardware to do so. XFS has also had a history for randomly corrupting data. While this might have improved over time, I don't trust it. XFS does have dynamic inode allocation, and better data storage algorithms than the Ext-family. It's also a good performer, but Ext4 give XFS a run for its money. -- . O . O . O . . O O . . . O . . . O . O O O . O . O O . . O O O O . O . . O O O O . O O O signature.asc Description: OpenPGP digital signature
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Volkan YAZICI put forth on 7/27/2010 8:22 AM: You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. [1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html 1. Never quote forum or email posts as empirical or reliable evidence of anything. They are opinion, unless they quote fact from reliable sources to back that opinion (which I did in my original post). There are of course exceptions to this rule of thumb, the main one being when the post is made by a developer who is a recognized authority on the piece of software being discussed. In the case of XFS this would be Dave Chinner, Alex Elder, Eric Sandeen, Christoph Hellwig, and others. In the case of EXT2/3/4 this would be Ted Tso, who happens, BTW, to work hand in hand with the XFS developers because they rely on each others patches to other parts of the Linux kernel. Not the last two entries: https://kerneltrap.org/mailarchive/linux-fsdevel/2010/7/16/expand In the case of Postfix this would be Wietse Venema and Viktor Duchovni. In the case of Linux this would be Linus Torvalds, Marcelo Tosatti, Alan Cox, Andrew Morton and others. Etc. 2. If you are going to quote opinions from unreliable sources, at least read the entire thread before quoting it. In this case, again, your source contradicts what you state, and then, oddly, himself: PS. Apparently they've improved some power-failure problems with XFS since I used it, but then they also said that *before* I started using it so I can't say I'd put any trust in that. Aneurin Price admits he is aware of new changes that fix the problem, but then discounts the fact and gives an anecdotal story as to why the fix isn't really a fix. [2] http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F You didn't even read the information you linked to, which contradicts what you state: The fix for this issue has been in mainline since May 2007, over 3 years ago. Q: Why do I see binary NULLS in some files after recovery when I unplugged the power? Update: This issue has been addressed with a CVS fix on the 29th March 2007 and merged into mainline on 8th May 2007 for 2.6.22-rc1. And considering my personal experiences, reiserfs is the fastest fs (among ext3 and xfs) in terms of boot recovery phase times. So, given that XFS recovery of any size filesystem is less than one second, reiserfs recovery is so much less than 1 second that you notice the difference? From: http://www.enterprisenetworkingplanet.com/netos/article.php/623661/XFS-Its-worth-the-wait.htm XFS also provides file system journaling. This means that XFS uses database recovery techniques to recover a consistent file system state after a system crash. Using journaling, XFS is able to accomplish this recovery in under a second, regardless of the file system size. [1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html Here you quoted misinformation, because the opinions were based on experience the OPs had with the software before it was patched to fix the problem, i.e. more than 3 years ago. You treated this as empirical evidence in your argument against XFS. I have shown this to be incorrect. [2] http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F You quoted this FAQ item solely based on the tile, without reading it, in your effort to denounce XFS. The article clearly states the problem was fixed over 3 years ago, which you conveniently ignored. From now on, please get your facts straight, with proper documentation, before trying to denounce a fantastic piece of FOSS into which many top-of-their-game kernel engineers have put tens of thousands of man hours, striving to make it the best it can be--and are wildly succeeding. Join the xfs mailing list and you might learn something useful in place of this trash you're talking about it. Better yet, read what Hans Reiser had to say about XFS. He was totally enamored with it, and wanted to duplicate many of its features: http://www.osnews.com/story/69 Hans Reiser: XFS is an excellent file system, and there is an important area where XFS is higher performance than we are...ReiserFS does a complete tree traversal for every 4k block it writes, and then it inserts one pointer at a time into the tree, which means that every 4k write incurs the overhead of a balancing of the tree (which means it moves data around). For this reason, XFS has better very large file performance...If you want to stream multi-media data for Hollywood style applications, or use ACLs now rather than wait for Reiser4, you might want to use XFS. This is an area we are still experimenting with. We currently do what ext2 does, and preallocate blocks. What XFS does is much better, they
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tuesday 27 July 2010 09:51:53 Stan Hoeppner wrote: Lisi put forth on 7/27/2010 2:23 AM: On Tuesday 27 July 2010 08:10:15 Stan Hoeppner wrote: XFS which is superior to all other Linux filesystems. Stan - Have you the time to give a rationale for this? Sure. Thanks, Stan, for a lucid and erudite exposition. Much appreciated. Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201007271731.34069.lisi.re...@gmail.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 6:19 PM, Stan Hoeppner s...@hardwarefreak.comwrote: Volkan YAZICI put forth on 7/27/2010 8:22 AM: You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. [1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html a fantastic piece of FOSS into which many top-of-their-game kernel engineers have put tens of thousands of man hours, striving to make it the best it can be--and are wildly succeeding. That's was very informative, thanks. You got me curious and I will test XFS on my home system. To be honest I am still little wary of using XFS in a production environment. For years now I have heard stories of power failures with catastrophic results when using XFS. Anyone who using XFS in a mission critical production environment? Anyone has experience with that?
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Aaron Toponce put forth on 7/27/2010 10:41 AM: XFS has also had a history for randomly corrupting data. While this might have improved over time, I don't trust it. Can you cite or reference anything to back your claim? Time frame? Irix or Linux? Serious users reported this or casual/hobbyist users? If this was ever the case the situation could not have lasted long before patches fixed it. Have you seen SGI's customer list and the size of the systems and storage they run with nothing but XFS? For instance, NAS has over 1.4PB of XFS filesystems, 1PB CXFS and over 400TB XFS: http://www.nas.nasa.gov/Resources/Systems/columbia.html http://www.nas.nasa.gov/Resources/Systems/archive_storage.html NASA trusts it with over 1PB of storage, but _you_ don't trust it? Who are you again? How many hundreds of TB of storage do you manage on EXT3/4? ;) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f1551.8010...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: Volkan YAZICI put forth on 7/27/2010 8:22 AM: 1. Never quote forum or email posts as empirical or reliable evidence of anything. You're right, my bad. You quoted this FAQ item solely based on the tile, without reading it, in your effort to denounce XFS. The article clearly states the problem was fixed over 3 years ago, which you conveniently ignored. I read the very same sentence, but AFAIK, default kernel for xfs bundled with lenny doesn't have that fix. From now on, please get your facts straight, with proper documentation, before trying to denounce a fantastic piece of FOSS into which many top-of-their-game kernel engineers have put tens of thousands of man hours, striving to make it the best it can be--and are wildly succeeding. Join the xfs mailing list and you might learn something useful in place of this trash you're talking about it. About a year ago, in a similar rush to yours, I ported two of our PostgreSQL database servers to XFS. During testing period, I even couldn't *recover* the / fs after the very first power failure test. Whole testing period took 1 week and the result was negative. This is my experience with XFS, and not much more thrash than your technical knowledge. And instead of being a technology zealot, you'd be better put forward some real world case scenarios. Try unplugging your xfs machineS that many timeS, and let's discuss this topic again. Yep, my findings might be deprecated, but I don't know any others investigating the same subject with recent versions. BTW, I still couldn't understand your temper and rudeness. I just share my experience, and try it, it works. Regards. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87sk34u9j7@alamut.alborz.net
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On 7/27/2010 11:20 AM, Stan Hoeppner wrote: Aaron Toponce put forth on 7/27/2010 10:41 AM: XFS has also had a history for randomly corrupting data. While this might have improved over time, I don't trust it. Can you cite or reference anything to back your claim? Time frame? Irix or Linux? Serious users reported this or casual/hobbyist users? If this was ever the case the situation could not have lasted long before patches fixed it. Have you seen SGI's customer list and the size of the systems and storage they run with nothing but XFS? For instance, NAS has over 1.4PB of XFS filesystems, 1PB CXFS and over 400TB XFS: We have used it three times in the past, and lost about 5TB worth of data due to corruption. The data corruption appeared to not be the result of lost power to the drive. Imperical evidence is enough for me to stop trusting it. I've also had friends who are admins that have complained of XFS data corruption, mainly with regards to booting. I don't know their specific scenarios, but they stopped using XFS as well. NASA trusts it with over 1PB of storage, but _you_ don't trust it? Who are you again? How many hundreds of TB of storage do you manage on EXT3/4? ;) I guess NASA has us beat. Nothing in the PB range, that's for sure. Currently, at my location, we have about 40 TB of SAN, with another 50 TB on the way. In production, we have about 200 TB SAN. We'll be building a federated shadowing infrastructure that well have Oracle databases in 16 different locations across the United States. We're currently targeting about 20 TB in each of the 16 locations. We won't be deploying XFS. -- . O . O . O . . O O . . . O . . . O . O O O . O . O O . . O O O O . O . . O O O O . O O O signature.asc Description: OpenPGP digital signature
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Aniruddha put forth on 7/27/2010 9:43 AM: On Tue, Jul 27, 2010 at 3:22 PM, Volkan YAZICI yazic...@ttmail.com wrote: You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. Ext3 has the same problems when not properly configured: Ext3 does not do checksumming when writing to the journal. If barrier=1 is not enabled as a mount option (in /etc/fstab), and if the hardware is doing out-of-order write caching, one runs the risk of severe filesystem corruption during a crash. http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal For the record I use ext3, I remember XFS as not being reliable enough (with power failures etc). This isn't a filesystem problem, or a kernel problem, or any other technical problem. This is a user problem. You will _never_ get computing technology the fully does what you _think_ it should upon loss of power. Period. XFS will prevent filesystem corruption (lookup the definition) but it will not prevent data loss. These are two completely different things. _No_ filesystem will fully prevent data loss when power is lost, but most will prevent filesystem corruption. Again, these are two different things. If you want maximum performance, you have to enable drive caches. Doing so causes more data loss when the power goes, and again, it's not the fault of the filesystem. If you want maximum protection against data loss, you have to disable drive caches, reduce the size of the in memory journal log buffer, etc, etc. Doing all of these things will absolutely murder your FS performance. This is a balancing act folks. You can't have your cake and eat it too. I'd also like to add that anyone smart enough to be on this list is smart enough to know you should have a UPS, regardless of what filesystem you use. If you're not you shouldn't be here. If you disagree on the technical merits (not cost), you're uneducated and/or stubborn. If you disagree on a cost basis, your data isn't valuable, period. A decent low end UPS for a desktop system that will get you through all brown outs and far enough through a storm outage (15-30 minutes) to do a proper shutdown costs about $50 USD. That's less than a carton of cigarettes in New York City, less than 3 regular price large pizzas at Dominos, and $25 less than a tank of gas for a full size pickup, which would last most people one week of commute. The cost of one tank of gas for 3-5 years of power protection before needing a battery replacement. I guess I should evangelize UPS as much as XFS given the benefits. Except XFS is free. ;) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f17db.7010...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: NASA trusts it with over 1PB of storage, but _you_ don't trust it? Who are you again? How many hundreds of TB of storage do you manage on EXT3/4? ;) NASA also trusts Windows and NTFS too? Who are you again? I think you are confusing apples and oranges. Everbody's requirements might differ, and hence do their tools. Instead of being a tech zealot, one just need to choose the right tool for the right job. I don't even think Linus is using XFS too. Isn't he a technical person in terms of your definition? So what should we do in that case? Ask to RMS? Regards. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87ocdsu93o@alamut.alborz.net
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue July 27 2010, Stan Hoeppner wrote: I'd also like to add that anyone smart enough to be on this list is smart enough to know you should have a UPS, regardless of what filesystem you use. If you're not you shouldn't be here. If you disagree on the technical merits (not cost), you're uneducated and/or stubborn. If you disagree on a cost basis, your data isn't valuable, period. A decent low end UPS for a desktop system that will get you through all brown outs and far enough through a storm outage (15-30 minutes) to do a proper shutdown costs about $50 USD. That's less than a carton of cigarettes in New York City, less than 3 regular price large pizzas at Dominos, and $25 less than a tank of gas for a full size pickup, which would last most people one week of commute. The cost of one tank of gas for 3-5 years of power protection before needing a battery replacement. this is something that I preach to EVERYONE who has a computer. Some people don't understand that I leave my computer plugged in running 24/7. they give me a deer-in-the-headlights look when I tell them I don't turn my computer off. But I live in Georgia, home of MASSIVE thunderstorms. I also live at the end of a street with 110 foot tall oak pine trees along side the road, and right next to our electric poles. In 5 years we have had 3 trees drop on wires cause loss of power, and I've had up to 20+ entries in the apcupsd.log file in ONE day, from thunder boomers. I have THREE UPSes in my house, not just my PC, but ALL electronic equipment, TV, stero AND Dish satellite receiver. I would NEVER plug anything electronic in, in MY house WITHOUT an UPS. I guess I should evangelize UPS as much as XFS given the benefits. Except XFS is free. ;) UPSes are really cheap and EXCELLENT insurance for not only your hardware, but your DATA. I won't even mention a company, and I DON'T work for them! -- Paul Cartwright Registered Linux user # 367800 Registered Ubuntu User #12459 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201007271342.01889.deb...@pcartwright.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 10:29 AM, Volkan YAZICI yazic...@ttmail.com wrote: On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: NASA trusts it with over 1PB of storage, but _you_ don't trust it? Who are you again? How many hundreds of TB of storage do you manage on EXT3/4? ;) NASA also trusts Windows and NTFS too? NASA also backs up their data on 5.25 floppy disks [1]. [1] *completely made up information
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
We use XFS in production at work. Where I work, we are routinely dealing with hundreds of terabytes of data (I have heard the word petabyte bandied about in several meetings), so we are beyond or hovering on the edge of the size limits and performance limits of the ext filesystems. At home, I primarily do reiserfs, for the simple reason that I have had need in the past (more than one would guess) where I have needed to shrink a filesystem. In fact, I needed to do so on a box at work. Right now, I am trying to get my brain around the improvements in btrfs, and hoping that will take off as many say it will. On Tue, Jul 27, 2010 at 1:03 PM, Aniruddha mailingdotl...@gmail.com wrote: On Tue, Jul 27, 2010 at 6:19 PM, Stan Hoeppner s...@hardwarefreak.comwrote: Volkan YAZICI put forth on 7/27/2010 8:22 AM: You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. [1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html a fantastic piece of FOSS into which many top-of-their-game kernel engineers have put tens of thousands of man hours, striving to make it the best it can be--and are wildly succeeding. That's was very informative, thanks. You got me curious and I will test XFS on my home system. To be honest I am still little wary of using XFS in a production environment. For years now I have heard stories of power failures with catastrophic results when using XFS. Anyone who using XFS in a mission critical production environment? Anyone has experience with that?
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: I'd also like to add that anyone smart enough to be on this list is smart enough to know you should have a UPS, regardless of what filesystem you use. If you're not you shouldn't be here. If you disagree on the technical merits (not cost), you're uneducated and/or stubborn. You are still not getting it, don't you? We have thousands of embedded linuxes in the wild and they are just simple data aggragetors. You can't have a power backup unit in such a condition. I'd also like to add that anyone smart enough to be on this list is smart enough to know you cannot have a UPS for embedded systems, regardless of what filesystem you use. If you're not you shouldn't be here. If you disagree on the technical merits (not cost), you're uneducated and/or stubborn. Regards. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k4ogu7on@alamut.alborz.net
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Dne, 27. 07. 2010 19:48:53 je Mark napisal(a): NASA also trusts Windows and NTFS too? NASA also backs up their data on 5.25 floppy disks [1]. [1] *completely made up information NASA as an authority on reliable storage? C'mon, the bozos can't even be trusted with their own Challengers, Columbias, OR the astronauts therein! -- Regards, Klistvud Certifiable Loonix User #481801 http://bufferoverflow.tiddlyspot.com -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1280254750.196...@compax
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Aniruddha put forth on 7/27/2010 12:03 PM: On Tue, Jul 27, 2010 at 6:19 PM, Stan Hoeppner s...@hardwarefreak.comwrote: Volkan YAZICI put forth on 7/27/2010 8:22 AM: You are missing a very important point: Durability to power failures. (Excuse me, but a majority of GNU/Linux users are not switched to a UPS or something.) And that's where XFS totally fails[1][2]. [1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html a fantastic piece of FOSS into which many top-of-their-game kernel engineers have put tens of thousands of man hours, striving to make it the best it can be--and are wildly succeeding. That's was very informative, thanks. You got me curious and I will test XFS on my home system. To be honest I am still little wary of using XFS in a production environment. For years now I have heard stories of power failures with catastrophic results when using XFS. Anyone who using XFS in a mission critical production environment? Anyone has experience with that? How about, and this will probably shock many of you: 1. Kernel.org All of Linux source, including what becomes the Debian kernel, and the kernels of all other Linux distros, is served from XFS filesystems: A bit more than a year ago (as of October 2008) kernel.org, in an ever increasing need to squeeze more performance out of it's machines, made the leap of migrating the primary mirror machines (mirrors.kernel.org) to XFS. We site a number of reasons including fscking 5.5T of disk is long and painful, we were hitting various cache issues, and we were seeking better performance out of our file system. After initial tests looked positive we made the jump, and have been quite happy with the results. With an instant increase in performance and throughput, as well as the worst xfs_check we've ever seen taking 10 minutes, we were quite happy. Subsequently we've moved all primary mirroring file-systems to XFS, including www.kernel.org , and mirrors.kernel.org. With an average constant movement of about 400mbps around the world, and with peaks into the 3.1gbps range serving thousands of users simultaneously it's been a file system that has taken the brunt we can throw at it and held up spectacularly. 2. NASA Advanced Supercomputing Facility, NASA Ames Research Center See my other post for details 3. Industrial Light and Magic -- ILM At one time ILM had one of the largest installed SGI SAN storage systems on the planet, may have been _the_ largest, running XFS. It backed their render farm(s). They don't currently have any render system info on their site that I can find. Given the number, size, and scope of their animation projects and the size to which their rendering farm has grown, they may have very well switched SAN vendors over the years. I don't know if they still use XFS or not. I would think so given that they're working with multi hundred gigabyte files daily. Many, many others. What you have to understand is that XFS has been around a long long time, 17 years in both IRIX and Linux. It's older than EXT2. Back before cheap Intel/AMD clusters took over the supercomputing marketplace, SGI MIPS IRIX systems with XFS owned upwards of 30-40% of that market. XFS in various platforms and versions has been in government labs, corporations and academia for over a decade. At one time Prof Stephen Hawking had his own personal 32 CPU SGI Origin 3800 for running cosmology calculations in order to prove his theories. It had XFS filesytems, as have all SGI systems since 1994. Here's a list of organizations that have volunteered information to xfs.org. It is by far not a complete list, and most of the major SGI customers with XFS on huge SAN systems aren't listed. Note NAS at NASA Ames isn't listed. http://xfs.org/index.php/XFS_Companies -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f25d5.1040...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Stan Hoeppner writes: I'd also like to add that anyone smart enough to be on this list is smart enough to know you should have a UPS, regardless of what filesystem you use. If you're not you shouldn't be here. I guess I don't belong here then... -- John Hasler -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87ocdsrdt0@thumper.dhh.gt.org
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Volkan YAZICI put forth on 7/27/2010 12:19 PM: About a year ago, in a similar rush to yours, I ported two of our PostgreSQL database servers to XFS. During testing period, I even couldn't *recover* the / fs after the very first power failure test. What write operations were you performing at the time you pulled the plug? Unless you were writing the superblock it'd be almost impossible to hose the filesystem to the point it couldn't mount. Were you doing a resize operation when you pulled the plug? xfs_growfs? As far as recovery, it's automatic upon mounting the XFS filesystem. What do you mean, precisely, by couldn't *recover* the / fs? And apologies if my tone seemed rude. I'm basically a one man army trying to defeat misinformation WRT XFS and attempt to educate ppl with the correct information. I guess I'm feeling outnumbered and thus being more aggressive. Again, apologies. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f27d6.1070...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Volkan YAZICI put forth on 7/27/2010 12:29 PM: I don't even think Linus is using XFS too. Isn't he a technical person Linus uses them all. You should know that. in terms of your definition? So what should we do in that case? Ask to RMS? kernel.org servers all run XFS as well. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f299a.9090...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: What write operations were you performing at the time you pulled the plug? Unless you were writing the superblock it'd be almost impossible to hose the filesystem to the point it couldn't mount. Were you doing a resize operation when you pulled the plug? xfs_growfs? As far as recovery, it's automatic upon mounting the XFS filesystem. What do you mean, precisely, by couldn't *recover* the / fs? Vanilla XFS with noatime,notail like basic mount options. The test was simple, I was just typing SELECT 1 from a psql command line (this query shouldn't even hit to disk, it just basically returns 1) and unplugged machine. At boot, I dropped to fsck command line. At command prompt, I manually fiddled around with fsck of xfs to recover the unmounted / filesystem, but had no luck. (I also tried recommendations and informative messages supplied by manpages and command outputs/warnings.) Also if you would Google, it shouldn't be hard to spot similar experiences from other people. At NASA, they might have genius technicians; but, IMHO a majority of the linux users would want a filesystem to recover without a prompt from the user. I'm basically a one man army trying to defeat misinformation WRT XFS and attempt to educate ppl with the correct information. I am glad -users ml have you; and I'd be really, really appreaciated if somebody having experience and knowledge on fs issues can shed some light to our ignorance. I also support the replacement of default fs with something that is much more recent. From this point of view, XFS is a superior alternative. You are totally right with your claims about its advantages over other alternatives. But as you can see, people still complain about XFS's sensitivity to power failures. Assuming a majority of your users aren't behind a UPS, you can sell/ship your product with such a default filesystem choice. But as you said, there are no published concrete benchmarks about this issue. It is all what people claim in the mailing lists. If you would share some of your findings about Power Failures and XFS to convince us, I'm sure most of us will be happy to advocate XFS's this achievement. Regards. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87fwz4u4oi@alamut.alborz.net
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Volkan YAZICI put forth on 7/27/2010 12:59 PM: On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes: I'd also like to add that anyone smart enough to be on this list is smart enough to know you should have a UPS, regardless of what filesystem you use. If you're not you shouldn't be here. If you disagree on the technical merits (not cost), you're uneducated and/or stubborn. You are still not getting it, don't you? We have thousands of embedded I get it now. This is the first you mentioned embedded systems. linuxes in the wild and they are just simple data aggragetors. You can't have a power backup unit in such a condition. Sure you can: http://www.embeddedarm.com/products/board-detail.php?product=TS-BAT3 And they make 'em even much smaller than that. If you get the right books, _you_ can make 'em smaller than that. :) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f3082.8090...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Time for some testing, I will put Debian stable with XFS on my laptop and see how well it deals with power failures :)
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Volkan YAZICI put forth on 7/27/2010 2:04 PM: unplugged machine. At boot, I dropped to fsck command line. At command Were you forced to the command line or did you manually select to go to the command line? It sounds like you chose to, not forced to. prompt, I manually fiddled around with fsck of xfs to recover the unmounted / filesystem, but had no luck. Did you read the xfs documentation before embarking on this power loss experiment? Or did you it should just work regardless of your actions, or lack of action? It sounds like you ran xfs_repair on a filesystem in an inconsistent state and forced changes, which is a no-no. (I also tried recommendations and informative messages supplied by manpages and command outputs/warnings.) Also if you would Google, it shouldn't be hard to spot similar experiences from other people. I'm guessing most of them didn't look before taking the XFS leap. At NASA, they might have genius technicians; but, IMHO a majority of the linux users would want a filesystem to recover without a prompt from the user. So the system wouldn't boot and you were dropped to a prompt. You manually fiddled around with fsck of xfs and made no progress. It would be nice to have seen all of that at the time. What were your results when you did this same power yank test with ext2/3, ReiserFS, and the other filesystems you tested in this way? I'm basically a one man army trying to defeat misinformation WRT XFS and attempt to educate ppl with the correct information. I am glad -users ml have you; and I'd be really, really appreaciated if somebody having experience and knowledge on fs issues can shed some light to our ignorance. I also support the replacement of default fs with something that is much more recent. From this point of view, XFS is a superior alternative. You are totally right with your claims about its advantages over other alternatives. But as you can see, people still complain about XFS's sensitivity to power failures. Assuming a majority of your users aren't behind a UPS, you can sell/ship your product with such a default filesystem choice. But as you said, there are no published concrete benchmarks about this issue. It is all what people claim in the mailing lists. If you would share some of your findings about Power Failures and XFS to convince us, I'm sure most of us will be happy to advocate XFS's this achievement. I've tried to dig up accurate accounts of the power loss corruption issue post 2007 (when it was supposed to have been fixed) a few times but couldn't find anything concrete enough to be worth referencing. I freely admit I've done no power loss testing of XFS myself. This probably has to do with the fact that I'm a firm believer in orderly shutdowns and redundant power, and that I don't really have any test systems available. I'll ask around on the XFS list and see what folks have to say. I'm somewhat interested in seeing where BTRFS is in 2-3 years. It may be stable enough for production by then, and should be as fast or faster than XFS on some workloads. Maybe it'll even handle sudden power loss gracefully. :) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f3ce4.2080...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Aniruddha put forth on 7/27/2010 2:47 PM: Time for some testing, I will put Debian stable with XFS on my laptop and see how well it deals with power failures :) Thanks for picking up the torch/gauntlet/whatever. How do you laptop test this issue? It's a laptop. Yank the battery? Yanking the wal-wart (transformer) won't do diddly. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f3dd8.9070...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 10:13 PM, Stan Hoeppner s...@hardwarefreak.com wrote: Aniruddha put forth on 7/27/2010 2:47 PM: Time for some testing, I will put Debian stable with XFS on my laptop and see how well it deals with power failures :) Thanks for picking up the torch/gauntlet/whatever. How do you laptop test this issue? It's a laptop. Yank the battery? Yanking the wal-wart (transformer) won't do diddly. I'll use it 'till runs out of battery. :) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktimnqtfz4ozsbym+28wcotynoufhgrqea+bep...@mail.gmail.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 11:06 PM, Aniruddha mailingdotl...@gmail.com wrote: On Tue, Jul 27, 2010 at 10:13 PM, Stan Hoeppner s...@hardwarefreak.com wrote: Aniruddha put forth on 7/27/2010 2:47 PM: Time for some testing, I will put Debian stable with XFS on my laptop and see how well it deals with power failures :) Thanks for picking up the torch/gauntlet/whatever. How do you laptop test this issue? It's a laptop. Yank the battery? Yanking the wal-wart (transformer) won't do diddly. I'll use it 'till runs out of battery. :) Or even better, I'll use virtualbox to see what happens when I plug the power. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlk...@kemojxop36vj-wtcr7910eowbfm0bu9ejz@mail.gmail.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On Tue, Jul 27, 2010 at 01:39:18PM -0500, Stan Hoeppner wrote: Volkan YAZICI put forth on 7/27/2010 12:19 PM: About a year ago, in a similar rush to yours, I ported two of our PostgreSQL database servers to XFS. During testing period, I even couldn't *recover* the / fs after the very first power failure test. What write operations were you performing at the time you pulled the plug? Unless you were writing the superblock it'd be almost impossible to hose the filesystem to the point it couldn't mount. Were you doing a resize operation when you pulled the plug? xfs_growfs? As far as recovery, it's automatic upon mounting the XFS filesystem. What do you mean, precisely, by couldn't *recover* the / fs? Some anecdotal evidence in support of ext3's resilience to power loss: I recently lost power while my system was running. When power was restored, an fsck was automatically performed. During that fsck, I lost power again! I thought for sure I'd be hosed, but after the power came back and an fsck completed, everything seems to be working normally. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100727213610.ga14...@aurora.owens.net
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
Rob Owens put forth on 7/27/2010 4:36 PM: On Tue, Jul 27, 2010 at 01:39:18PM -0500, Stan Hoeppner wrote: Volkan YAZICI put forth on 7/27/2010 12:19 PM: About a year ago, in a similar rush to yours, I ported two of our PostgreSQL database servers to XFS. During testing period, I even couldn't *recover* the / fs after the very first power failure test. What write operations were you performing at the time you pulled the plug? Unless you were writing the superblock it'd be almost impossible to hose the filesystem to the point it couldn't mount. Were you doing a resize operation when you pulled the plug? xfs_growfs? As far as recovery, it's automatic upon mounting the XFS filesystem. What do you mean, precisely, by couldn't *recover* the / fs? Some anecdotal evidence in support of ext3's resilience to power loss: I recently lost power while my system was running. When power was restored, an fsck was automatically performed. During that fsck, I lost power again! I thought for sure I'd be hosed, but after the power came back and an fsck completed, everything seems to be working normally. If no writes are in process or pending, it really doesn't matter which fs you use, as none of them will suffer negative effects. All will recover after a journal replay or fsck. For journaling filesystems that are configured properly, even if a write is in progress when the power goes out, the filesystem will not be corrupted as a result. You will have lost data, but the filesystem itself will be healthy. This is one of the two main purposes of a journaling filesystem. The other is rapid recovery. You experienced both in your example. :) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c4f6e38.9080...@hardwarefreak.com
Re: Linux filesystems was [Re: Debian cd supporting ext4.]
On 20100727_134650, Stan Hoeppner wrote: Volkan YAZICI put forth on 7/27/2010 12:29 PM: I don't even think Linus is using XFS too. Isn't he a technical person Linus uses them all. You should know that. in terms of your definition? So what should we do in that case? Ask to RMS? kernel.org servers all run XFS as well. -- Stan Stan, Have you ever heard of the term 'invincible ignorance'? It is a term used in Catholic apologesis to describe the state of unbelievers who simply refuse to accept any argument intended to convert them. I suggest that you come to recognize some invincible ignorance here. Also, you have asserted with some vigor upstream, that any computer person who doesn't have UPS on all his/her systems is somehow not really a real computer person. Personally, I am puzzled about a test of XFS on a system running a serious relational database managed using PostgreSQL that does not have UPS. I infer that there was no UPS because if there were, pulling the plug would not have caused any harm, or, knowing that there was a UPS the testers would have not pulled the plug because it would have been a silly 'test'. Your original post in this thread addressed a quite disfunctional attitude of OP, and IMHO, was correct but somewhat harshly worded. In truth, he simply cannot have everything he wants all at the same time. You should have left it at that, IMHO. I suggest that you try to learn to live with the fact that there is a vast unwashed mass of invincibly ignorant people who call themselves computer experts. Unless you relish this heated controversy, you should limit your recommendation of XFS to shops that take for granted the necessity of having UPS first. Peace. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100728015343.ga5...@big.lan.gnu