Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Saturday 16 February 2008 05:33:43 Wael Nasreddine wrote: Thank you for your detailed answer it helped a lot (Why was it necessary to quote the whole of it again?) please take a look at the file attached... and if you have any more suggestions please do tell me. Just a tiny point: you don't need defaults if another value appears as well, as in: /dev/system/home/home reiserfsdefaults,user_xattr 0 0 This will do just as well: /dev/system/home/home reiserfsuser_xattr 0 0 Defaults are what you get if you don't specify anything. Thank you all for this interesting discussion, which has given me a few ideas for improving --sync time on my server box. -- Rgds Peter -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
This One Time, at Band Camp, Peter Humphrey [EMAIL PROTECTED] said, On Sat, Feb 16, 2008 at 10:35:18AM +: On Saturday 16 February 2008 05:33:43 Wael Nasreddine wrote: Thank you for your detailed answer it helped a lot (Why was it necessary to quote the whole of it again?) please take a look at the file attached... and if you have any more suggestions please do tell me. Just a tiny point: you don't need defaults if another value appears as well, as in: /dev/system/home /home reiserfsdefaults,user_xattr 0 0 This will do just as well: /dev/system/home /home reiserfsuser_xattr 0 0 Defaults are what you get if you don't specify anything. Thank you all for this interesting discussion, which has given me a few ideas for improving --sync time on my server box. Thanks for the TIP and again for your help :) -- Wael Nasreddine http://wael.nasreddine.com PGP: 1024D/C8DD18A2 06F6 1622 4BC8 4CEB D724 DE12 5565 3945 C8DD 18A2 .: An infinite number of monkeys typing into GNU emacs, would never make a good program. (L. Torvalds 1995) :. pgpWGgXi0hQ5c.pgp Description: PGP signature
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Sat, 2008-02-16 at 06:33 +0100, Wael Nasreddine wrote: it's done, thanks, BTW what's your home partition FS? your choice is ext3 or reiserFS?? I use reiserfs3.6 without notail but that doesn't mean that it would be a good choice for you. I'm on laptop and disk space efficiency is a big topic for me so I use tail-packing wherever suitable. And yes, I am a fan of ReiserFS-3.6. I think it's the best multipurpose FS. You can easily adapt it for high performance or high disk space efficiency. If its journaling would be as good as Ext3's data=journal I'd use it everywhere except for small partitions (ext2) and big files (ext3 and xfs). One last thing, since I'm on LVM resizing the partition is a must feature, in ext3 I use resize2fs which works quite nicely, is resize_reiserfs as reliable as resize2fs is?? Yes, it's just as good and the sky's the limit for resizing :) Oh, by the way: If you choose to use XFS somewhere, keep in mind that you can't shrink and XFS-FS. Neither online nor offline. One last thing: It's a bit old but I think it's still interesting, especially for XFS-users: http://everything2.com/index.pl?node_id=1479435 Thank you for your detailed answer it helped a lot, I just finished resizing/migrating all partitions, Though I still have the Storage partition, which is for my Mp3z and is almost 70Gb, with ext3, I'll see later if I do migrate to ReiserFS or not but the rest is done, please take a look at the file attached... and if you have any more suggestions please do tell me. You could use the noatime mount option on all your partitions. With atimes enabled, every time you read a file, its (mostly useless) access time is updated which results in a write action. The only program that I know to use atimes is mutt (for mail spools only). You could also take a look at the link I've posted in my last message. It contains useful mount options for XFS. signature.asc Description: This is a digitally signed message part
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
This One Time, at Band Camp, Florian Philipp [EMAIL PROTECTED] said, On Sat, Feb 16, 2008 at 06:22:13PM +0100: On Sat, 2008-02-16 at 06:33 +0100, Wael Nasreddine wrote: it's done, thanks, BTW what's your home partition FS? your choice is ext3 or reiserFS?? I use reiserfs3.6 without notail but that doesn't mean that it would be a good choice for you. I'm on laptop and disk space efficiency is a big topic for me so I use tail-packing wherever suitable. And yes, I am a fan of ReiserFS-3.6. I think it's the best multipurpose FS. You can easily adapt it for high performance or high disk space efficiency. If its journaling would be as good as Ext3's data=journal I'd use it everywhere except for small partitions (ext2) and big files (ext3 and xfs). One last thing, since I'm on LVM resizing the partition is a must feature, in ext3 I use resize2fs which works quite nicely, is resize_reiserfs as reliable as resize2fs is?? Yes, it's just as good and the sky's the limit for resizing :) Oh, by the way: If you choose to use XFS somewhere, keep in mind that you can't shrink and XFS-FS. Neither online nor offline. One last thing: It's a bit old but I think it's still interesting, especially for XFS-users: http://everything2.com/index.pl?node_id=1479435 Thank you for your detailed answer it helped a lot, I just finished resizing/migrating all partitions, Though I still have the Storage partition, which is for my Mp3z and is almost 70Gb, with ext3, I'll see later if I do migrate to ReiserFS or not but the rest is done, please take a look at the file attached... and if you have any more suggestions please do tell me. You could use the noatime mount option on all your partitions. With atimes enabled, every time you read a file, its (mostly useless) access time is updated which results in a write action. The only program that I know to use atimes is mutt (for mail spools only). You could also take a look at the link I've posted in my last message. It contains useful mount options for XFS. Thank you for the TIP as well, I added noatime to all partitions except for /home because mutt keeps imap cache on it, I'm not sure if it's atime depending or not I should probably check it out though... Thanks :) -- Wael Nasreddine http://wael.nasreddine.com PGP: 1024D/C8DD18A2 06F6 1622 4BC8 4CEB D724 DE12 5565 3945 C8DD 18A2 .: An infinite number of monkeys typing into GNU emacs, would never make a good program. (L. Torvalds 1995) :. pgpkjO97ncvV4.pgp Description: PGP signature
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Fri, 2008-02-15 at 14:36 +0100, Strong Cypher wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm looking for an alternative to ext2/3. I have put reiser3/4 out because of project seems to be off now ... or not really active I really want an active project. Is they a good fs that is extremly adapted to gentoo system (portage ...) Is they fs that support gzip like reiser4 do ? For exemple , with reiser4 the portage directory don't take a lot of space, and so read it it's really fast... I want a alternative is ext4 a good alternative ? Don't know about ext4 but for portage trees I found ext2 to be faster than everything else I tried (primarily reiserfs3.6). Have you taken a look at XFS or JFS? Concerning online compression I can only think of cramfs (which is read-only) or NTFS (do they support compression by now? I know that I can format a partition and set it to compressed but I've not tried it.) signature.asc Description: This is a digitally signed message part
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Fri, 15 Feb 2008 14:36:25 +0100, Strong Cypher wrote: For exemple , with reiser4 the portage directory don't take a lot of space, and so read it it's really fast... You could use a file, like this, then put ext2 on it. http://gentoo-wiki.com/TIP_Speeding_up_portage#Make_A_Sparse_File_to_create_portage_in -- Neil Bothwick In an atomic war, all men will be cremated equal. signature.asc Description: PGP signature
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Am Freitag, 15. Februar 2008 schrieb ext Strong Cypher: Is they a good fs that is extremly adapted to gentoo system (portage ...) Huh. Why should somebody write a filesystem with Gentoo portage in mind? Is they fs that support gzip like reiser4 do ? See http://parallel.vub.ac.be/~johan/compFUSEd/, it's an overlay filesystem, so you're not bound to any specific real filesystem. OTOH, harddisks are cheap nowadays. For exemple , with reiser4 the portage directory don't take a lot of space, and so read it it's really fast... The same is true for reiser3. I want a alternative Well, there are plenty: xfs, jfs, ... is ext4 a good alternative ? ext4 is in early develoment, as are other new filesystems (btrfs for example), so no it isn't (yet, but probably sooner than reiser4). If you're searching something that stores small files efficiently, like reiser3 does, btrfs comes close, if I remember right. Bye... Dirk -- Dirk Heinrichs | Tel: +49 (0)162 234 3408 Configuration Manager | Fax: +49 (0)211 47068 111 Capgemini Deutschland | Mail: [EMAIL PROTECTED] Wanheimerstraße 68 | Web: http://www.capgemini.com D-40468 Düsseldorf | ICQ#: 110037733 GPG Public Key C2E467BB | Keyserver: www.keyserver.net signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Dirk Heinrichs wrote: Am Freitag, 15. Februar 2008 schrieb ext Strong Cypher: For exemple , with reiser4 the portage directory don't take a lot of space, and so read it it's really fast... The same is true for reiser3. I want a alternative Well, there are plenty: xfs, jfs, ... Of the current main four FS's on modern linux, here's a general overview of them: ext3: Older, reliable, stable. You get very good tools support for ext3 (including online resize) and low cpu-usage for the most part but it's slower and less space efficient than more recent fs's. jfs: Better performance than ext3, deals with larger files reasonably well with low cpu usage. Not very commonly used to my knowledge. reiserfs (reiser3): very fast for most operations (the exception being directory creation iirc), especially efficient for dealing with many small files. It has noticeably higher cpu-usage than ext3/jfs. I believe there are also some potential performance bottlenecks on SMP systems as it makes liberal use of the Big Kernel Lock. xfs: high performance, especially when dealing with many large or small files; Gets along very well with raid arrays. Noticeably higher cpu usage than ext3/jfs. IIRC, it aggressively caches its writes so there is a slight possibility of data loss if your power goes out suddenly in the middle of a series of writes (I consider this a very small possibility, it is journalled like the other fs's on this list so the filesystem will still come up in a consistent state, you just may be missing some of the data you were writing). Very good online tools support provided with it. I'm sure someone will jump in to correct me if I've misremembered something. Favorite filesystems can be a bit like Window Manager or favorite Desktop debates. Aaron -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Aaron Clark wrote: Dirk Heinrichs wrote: Am Freitag, 15. Februar 2008 schrieb ext Strong Cypher: For exemple , with reiser4 the portage directory don't take a lot of space, and so read it it's really fast... The same is true for reiser3. I want a alternative Well, there are plenty: xfs, jfs, ... Of the current main four FS's on modern linux, here's a general overview of them: ext3: Older, reliable, stable. You get very good tools support for ext3 (including online resize) and low cpu-usage for the most part but it's slower and less space efficient than more recent fs's. jfs: Better performance than ext3, deals with larger files reasonably well with low cpu usage. Not very commonly used to my knowledge. reiserfs (reiser3): very fast for most operations (the exception being directory creation iirc), especially efficient for dealing with many small files. It has noticeably higher cpu-usage than ext3/jfs. I believe there are also some potential performance bottlenecks on SMP systems as it makes liberal use of the Big Kernel Lock. xfs: high performance, especially when dealing with many large or small files; Gets along very well with raid arrays. Noticeably higher cpu usage than ext3/jfs. IIRC, it aggressively caches its writes so there is a slight possibility of data loss if your power goes out suddenly in the middle of a series of writes (I consider this a very small possibility, it is journalled like the other fs's on this list so the filesystem will still come up in a consistent state, you just may be missing some of the data you were writing). Very good online tools support provided with it. I'm sure someone will jump in to correct me if I've misremembered something. Favorite filesystems can be a bit like Window Manager or favorite Desktop debates. Aaron Little addition to XFS, I tried it once a while ago. Every time the power failed, it would never boot again. I can say from personal experience and from what I have read from others, if you plan to use XFS, have a good UPS hooked up. It does not like power failures at all. YMMV Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Dale wrote: Little addition to XFS, I tried it once a while ago. Every time the power failed, it would never boot again. I can say from personal experience and from what I have read from others, if you plan to use XFS, have a good UPS hooked up. It does not like power failures at all. YMMV :) In the YMMV category, I've used XFS on pretty much every file server I've had in the last 4-5 years and it's never given me any trouble despite pretty much never having a UPS hooked up and a decent number of power outages. Granted, I never used it on my root filesystem, only storage partitions. Aaron -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Ok guy thanks for answer ... For my use, mix of ext2/ext3 in partition (lvm) and sparse file could speed up my system ... I think it could not at the same point of reiser4 but support in case of crash could really be better ... Thanks for answer It's not easy to create filesystem that's is perfect for all case I see ... perhaps one day :) (I can dream, or make it) 2008/2/15, Dale [EMAIL PROTECTED]: Aaron Clark wrote: Dale wrote: Little addition to XFS, I tried it once a while ago. Every time the power failed, it would never boot again. I can say from personal experience and from what I have read from others, if you plan to use XFS, have a good UPS hooked up. It does not like power failures at all. YMMV :) In the YMMV category, I've used XFS on pretty much every file server I've had in the last 4-5 years and it's never given me any trouble despite pretty much never having a UPS hooked up and a decent number of power outages. Granted, I never used it on my root filesystem, only storage partitions. Aaron Good idea not to use it on the / file system. LOL I was using Mandriva for my ex's Mom. After about three or four tries, I went back to reiserfs. It would crash but it would boot right back up again. Nothing lost that I know of. I just never trusted it again. I have also been told, and read elsewhere, that it is a pretty well known thing that it doesn't like power failures. It has its good points tho, which is why I was trying it out. Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Aaron Clark wrote: Dale wrote: Little addition to XFS, I tried it once a while ago. Every time the power failed, it would never boot again. I can say from personal experience and from what I have read from others, if you plan to use XFS, have a good UPS hooked up. It does not like power failures at all. YMMV :) In the YMMV category, I've used XFS on pretty much every file server I've had in the last 4-5 years and it's never given me any trouble despite pretty much never having a UPS hooked up and a decent number of power outages. Granted, I never used it on my root filesystem, only storage partitions. Aaron Good idea not to use it on the / file system. LOL I was using Mandriva for my ex's Mom. After about three or four tries, I went back to reiserfs. It would crash but it would boot right back up again. Nothing lost that I know of. I just never trusted it again. I have also been told, and read elsewhere, that it is a pretty well known thing that it doesn't like power failures. It has its good points tho, which is why I was trying it out. Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Friday 15 February 2008, Aaron Clark wrote: xfs: high performance, especially when dealing with many large or small files; Gets along very well with raid arrays. Noticeably higher cpu usage than ext3/jfs. IIRC, it aggressively caches its writes so there is a slight possibility of data loss if your power goes out suddenly in the middle of a series of writes (I consider this a very small possibility, it is journalled like the other fs's on this list so the filesystem will still come up in a consistent state, you just may be missing some of the data you were writing). Very good online tools support provided with it. Sigh. So many myths about journalled filesystems, so little time to squash them. ;-) First of all, you are contradicting yourself. First you say you think the possibility of data loss is slight, then you state at the end that some data loss may occur. The second part is right. Data losses are possible. Journalled filesystems do not prevent this. They deal with filesystem consistency. Second, no journalled filesystem in the whole wide world can prevent occurences of inconsisteny in case of a power cut. None, try as they might. Please commit the last two sentences to permanent memory. The reason for this isn't the cache in your computer's ram but the cache in modern harddrives. If the journal change still resides in the harddrive cache while your power cut occurs, bm - inconsistency. There is nothing a filesystem - journalled or not - can do about it. If you are really concerned about data loss and filesystem inconsistencies, use a good journalled fs *and* a small UPS that can shut your box down gracefully in case of a power cut. Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Friday 15 February 2008, Dale wrote: Little addition to XFS, I tried it once a while ago. Every time the power failed, it would never boot again. I can say from personal experience and from what I have read from others, if you plan to use XFS, have a good UPS hooked up. It does not like power failures at all. YMMV XFS was designed to be used in environments where the admin is supposed to GUARANTEE zero power outages. SGI built it for their mips machines doing cool stuff like video rendering. If you have a multi-million $ render farm churning out Hollywood's latest blockbuster complete with special effects, it is entirely reasonable to expect that the environment has UPS backup par excellence. So, using XFS on regular pcs without a UPS (or even with those dinky little 10 minute uptime jobs) is a gross misuse of the XFS technology IMNSHO. It simply was not built for that, in almost exactly the same way that Lamborghini did not build the Murcielago so you could nip down to the shops with it and buy a pack of fags... -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Friday 15 February 2008, Strong Cypher wrote: Ok guy thanks for answer ... For my use, mix of ext2/ext3 in partition (lvm) and sparse file could speed up my system ... I think it could not at the same point of reiser4 but support in case of crash could really be better ... Assuming you have chosen sane mkfs settings, the largest single file system improvement you could possibly ever make, is to split your filesystem up into various volumes according to their intended role. Like, /home and /var and /usr are separate filesystems. Ever other tweak you will ever make is miniscule in comparison to this one. Mind you, this is EXACTLY how Unix was designed to be used. Funny, that... -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Fri, 15 Feb 2008, Uwe Thiem wrote: Second, no journalled filesystem in the whole wide world can prevent occurences of inconsisteny in case of a power cut. None, try as they might. This is correct. If the journal change still resides in the harddrive cache while your power cut occurs, bm - inconsistency. But this isn't the reason. Harddrives know a flush command which - when properly used by the filesystem (and I guess reiserfs and ext3 use it properly) - forces the journal to be written before the actual change in the main file system occurs. Whence, no loss of consistency. [Of course, there are some harddrives which ignore the flush, but this should be counted as faulty hardware. Of course, on broken hardware, no software can work as it should.] If the power loss occurs *during* flushing the journal (and thus the journal might contain nonsense) the filesystem might still use a checksum over the journal to detect this and thus preserves consistency (although I don't know whether any existing filesystem currently does this). The real problem is that during power cut the harddrive might be writing complete nonsense *somewhere* - this is not related with any caching, and no software can safe you from this problem (and what is even worse is that there is no way to detect it...) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Suggestion: put your Portage and database trees on flash storage. I'd go with one of two routes: a fast USB stick or a quality CompactFlash card. At the moment, the one place I know of to get a quality CF card is NewEgg: they're selling a couple of 266x CF4-compliant cards, Transcend-branded. Addonics will be happy to sell you an adapter (~USD$30) that will go into a spare drive bay and turn the CF card into some really, really fast UDMA storage. (If it's not at least CF3-compliant, your CF storage will still work happily as an IDE hard drive, but it'll do it at PIO transfer rates, and you were looking for speed.) Putting both /usr/portage and /var/db on flash memory pulls it completely off any disk spindles that you'd otherwise have to share with /, or /usr, or whatever other filesystems you're likely to have on magnetic media. A word of warning, either way: don't put a Linux-native filesystem on any kind of flash memory. Wear leveling only works if the memory controller understands the filesystem you're writing to the drive. That means FAT16, or FAT32 if you're lucky. And, yes, I've tried to get information out of Transcend sales on whether or not they sell any products that speak alternative filesystems. I never got an answer back, which I think means, Ha ha ha! *wipes tears* That's funny! Ask another one! I'd ask, say, OCZ, but Transcend manufactured both of my OCZ USB flash drives. Still interested? You'll want about 2GB total: that seems to hold the entire current Portage tree, plus a good-sized /var/db, and leaves something like 500MB free for growth. Get 4GB if you're paranoid; it's cheap anyway. This assumes that you don't store /usr/portage/distfiles on the flash storage. I wouldn't, and didn't: make /usr/portage/distfiles a symlink to somewhere on your magnetic media, and make sure that the new directory (/usr/distfiles in my case) is owned by root:portage so that you can leave FEATURES=userfetch turned on in make.conf. For sanity reasons, you may want to mount your new FAT16/32 filesystem with -o uid=0,gid=0. Or, if you're using FEATURES=userpriv, maybe uid=250,gid=250 (portage:portage on my machine). That all depends on your particular FEATURE flags. On Fri, Feb 15, 2008 at 8:36 AM, Strong Cypher [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm looking for an alternative to ext2/3. I have put reiser3/4 out because of project seems to be off now ... or not really active I really want an active project. Is they a good fs that is extremly adapted to gentoo system (portage ...) Is they fs that support gzip like reiser4 do ? For exemple , with reiser4 the portage directory don't take a lot of space, and so read it it's really fast... I want a alternative is ext4 a good alternative ? -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.7 (GNU/Linux) iD8DBQFHtZVGEg3iyspSWPARAiitAJsGb87FwLBPir4a2y9NjSq+0uW9pgCfb7aW ZmCRw4wDqC4b/SBPumKY6kI= =16t6 -END PGP SIGNATURE- -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Fri, 2008-02-15 at 21:05 +0100, Wael Nasreddine wrote: Currently I have 2 partitions, a root and home partition, fortunately on LVM array, I was thinking of splitting them to /, /usr, /var, /home, /usr/portage, /mnt/storage the latter is to be used for Mp3z (around 12000) and movies... I was thinking of having the below filesystem schema: / : ext3 (-j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr: xfs (I never used it so please suggest mkfs.xfs options) /var: // /home : ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr/portage: ReiserFS (3? 4? options??) /mnt/storage: ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) Could you please comment/complete/change the schema above ?? I really would like to speed up my system a little bit, My system is entirely built on LVM array, and LVM is on DM-CRYPT so as you can see it's a quite slow due to the encryption... Oh one last thing, What do you suggest for a server? I have a Gentoo server and uptime can be over 5/6 months, everytime I reboot the server I have to manually scan the filesystem due to errors everywhere, any suggestions?? Thanks... First of all, if there are filesystem errors, check your cables, your controller and your disks. I don't think filesystem errors count as normal behavior ... To your filesystem scheme: Why do you use xfs for usr? AFAIK XFS is good at write speed but not worth the trouble when reading data and data in usr is usually written once, updated every few months and read many times a week (on rebooting Desktop PCs maybe once a day). I'd use reiserfs3.6, maybe even without notail to make it more space efficient. I'd also use ext2 on /usr/portage. These data don't need journaling. Everything's got an MD5-sum to make sure it's unchanged after a crash and you can easily resync. I found ext2 with 2k blocks to be faster than reiserfs3.6, even on read-performance. If I were you, I'd also use separate volumes for /tmp and /var/tmp (without ccache) with xfs. /home could use data=journal. Those data are precious and if I remember correctly, this setting even brings an obscure (i.e. undocumented) speed improvement with many parallel disk accesses, for example in a multi-user environment. signature.asc Description: This is a digitally signed message part
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Friday 15 February 2008 03:05:13 pm Wael Nasreddine wrote: Hey guys, Currently I have 2 partitions, a root and home partition, fortunately on LVM array, I was thinking of splitting them to /, /usr, /var, /home, /usr/portage, /mnt/storage the latter is to be used for Mp3z (around 12000) and movies... I was thinking of having the below filesystem schema: / : ext3 (-j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr: xfs (I never used it so please suggest mkfs.xfs options) /var: // /home : ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr/portage: ReiserFS (3? 4? options??) /mnt/storage: ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) This is from a very humbled ex-ext3 user... I finally decided to play around with reiserfs a while back and I have to tell you... I'll never go back to ext3 unless I really, really have to. The difference is easy to measure and pleasure once you make the move I've been setting up machines like this... /boot ext2 / reiserfs /home reiserfs /var reiserfs The difference in disk I/O is... nice!! and the reliability is the same as ext3. Untill the cold shoulder for reiser4 is thawed and it gets into the kernel source tree, I'd stay away from it for now however. Cheers. Could you please comment/complete/change the schema above ?? I really would like to speed up my system a little bit, My system is entirely built on LVM array, and LVM is on DM-CRYPT so as you can see it's a quite slow due to the encryption... Oh one last thing, What do you suggest for a server? I have a Gentoo server and uptime can be over 5/6 months, everytime I reboot the server I have to manually scan the filesystem due to errors everywhere, any suggestions?? Thanks... -- From the Desk of: Jerome D. McBride -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
This One Time, at Band Camp, Dale [EMAIL PROTECTED] said, On Fri, Feb 15, 2008 at 09:17:11AM -0600: Aaron Clark wrote: Dale wrote: Little addition to XFS, I tried it once a while ago. Every time the power failed, it would never boot again. I can say from personal experience and from what I have read from others, if you plan to use XFS, have a good UPS hooked up. It does not like power failures at all. YMMV :) In the YMMV category, I've used XFS on pretty much every file server I've had in the last 4-5 years and it's never given me any trouble despite pretty much never having a UPS hooked up and a decent number of power outages. Granted, I never used it on my root filesystem, only storage partitions. Aaron Good idea not to use it on the / file system. LOL I was using Mandriva for my ex's Mom. After about three or four tries, I went back to reiserfs. It would crash but it would boot right back up again. Nothing lost that I know of. I just never trusted it again. I have also been told, and read elsewhere, that it is a pretty well known thing that it doesn't like power failures. It has its good points tho, which is why I was trying it out. Dale :-) :-) Hey guys, Currently I have 2 partitions, a root and home partition, fortunately on LVM array, I was thinking of splitting them to /, /usr, /var, /home, /usr/portage, /mnt/storage the latter is to be used for Mp3z (around 12000) and movies... I was thinking of having the below filesystem schema: / : ext3 (-j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr: xfs (I never used it so please suggest mkfs.xfs options) /var: // /home : ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr/portage: ReiserFS (3? 4? options??) /mnt/storage: ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) Could you please comment/complete/change the schema above ?? I really would like to speed up my system a little bit, My system is entirely built on LVM array, and LVM is on DM-CRYPT so as you can see it's a quite slow due to the encryption... Oh one last thing, What do you suggest for a server? I have a Gentoo server and uptime can be over 5/6 months, everytime I reboot the server I have to manually scan the filesystem due to errors everywhere, any suggestions?? Thanks... -- Wael Nasreddine http://wael.nasreddine.com PGP: 1024D/C8DD18A2 06F6 1622 4BC8 4CEB D724 DE12 5565 3945 C8DD 18A2 .: An infinite number of monkeys typing into GNU emacs, would never make a good program. (L. Torvalds 1995) :. pgpawJQFt7veu.pgp Description: PGP signature
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Alois Hammer wrote: Suggestion: put your Portage and database trees on flash storage. There is no way I would do that or recommend it to anyone. Those devices have a very, very short life if written to frequently. Portage isn't a big problem because an emerge --sync will restore it - but database trees? You have to be kidding. Be lucky, Neil -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
This One Time, at Band Camp, Florian Philipp [EMAIL PROTECTED] said, On Fri, Feb 15, 2008 at 10:24:55PM +0100: On Fri, 2008-02-15 at 21:05 +0100, Wael Nasreddine wrote: Currently I have 2 partitions, a root and home partition, fortunately on LVM array, I was thinking of splitting them to /, /usr, /var, /home, /usr/portage, /mnt/storage the latter is to be used for Mp3z (around 12000) and movies... I was thinking of having the below filesystem schema: / : ext3 (-j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr: xfs (I never used it so please suggest mkfs.xfs options) /var: // /home : ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) /usr/portage: ReiserFS (3? 4? options??) /mnt/storage: ext3 (-m 0 -j -O dir_index,sparse_super,filetype) (Good mkfs options ??) Could you please comment/complete/change the schema above ?? I really would like to speed up my system a little bit, My system is entirely built on LVM array, and LVM is on DM-CRYPT so as you can see it's a quite slow due to the encryption... Oh one last thing, What do you suggest for a server? I have a Gentoo server and uptime can be over 5/6 months, everytime I reboot the server I have to manually scan the filesystem due to errors everywhere, any suggestions?? Thanks... First of all, if there are filesystem errors, check your cables, your controller and your disks. I don't think filesystem errors count as normal behavior ... I should check that out, thanks To your filesystem scheme: Why do you use xfs for usr? AFAIK XFS is good at write speed but not worth the trouble when reading data and data in usr is usually written once, updated every few months and read many times a week (on rebooting Desktop PCs maybe once a day). I'd use reiserfs3.6, maybe even without notail to make it more space efficient. I don't use XFS, curently I only have / and /home and I want to split it to more smaller partitions, I'm on LVM so it's easy, anyway I'm going with ReiserFS for /usr /var, would you please suggest mkfs.reiserfs options as I have nerver used ReiserFS-3 before (yep 5 years using linux and I've always used ext3...) also You didn't mention /var, would you say ReiserFS-3 is a good choice as well? I'd also use ext2 on /usr/portage. These data don't need journaling. Everything's got an MD5-sum to make sure it's unchanged after a crash and you can easily resync. I found ext2 with 2k blocks to be faster than reiserfs3.6, even on read-performance. I've already made the partition as suggested in [1] I used this command: $ mke2fs -b 1024 -N 20 -m 0 -O dir_index I guess 1K block size would be faster?? If I were you, I'd also use separate volumes for /tmp and /var/tmp (without ccache) with xfs. What did you mean by 'without ccache'? I have ccache and I use it... /home could use data=journal. Those data are precious and if I remember correctly, this setting even brings an obscure (i.e. undocumented) speed improvement with many parallel disk accesses, for example in a multi-user environment. it's done, thanks, BTW what's your home partition FS? your choice is ext3 or reiserFS?? One last thing, since I'm on LVM resizing the partition is a must feature, in ext3 I use resize2fs which works quite nicely, is resize_reiserfs as reliable as resize2fs is?? [1]: http://gentoo-wiki.com/TIP_Speeding_up_portage#Make_A_Sparse_File_to_create_portage_in -- Wael Nasreddine http://wael.nasreddine.com PGP: 1024D/C8DD18A2 06F6 1622 4BC8 4CEB D724 DE12 5565 3945 C8DD 18A2 .: An infinite number of monkeys typing into GNU emacs, would never make a good program. (L. Torvalds 1995) :. pgpZiQKZT26J8.pgp Description: PGP signature
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
On Sat, 2008-02-16 at 00:32 +0100, Wael Nasreddine wrote: To your filesystem scheme: Why do you use xfs for usr? AFAIK XFS is good at write speed but not worth the trouble when reading data and data in usr is usually written once, updated every few months and read many times a week (on rebooting Desktop PCs maybe once a day). I'd use reiserfs3.6, maybe even without notail to make it more space efficient. I don't use XFS, curently I only have / and /home and I want to split it to more smaller partitions, I'm on LVM so it's easy, anyway I'm going with ReiserFS for /usr /var, would you please suggest mkfs.reiserfs options as I have nerver used ReiserFS-3 before (yep 5 years using linux and I've always used ext3...) also You didn't mention /var, would you say ReiserFS-3 is a good choice as well? I don't think there's alot to do when creating a reiserfs. You could change the number of blocks for the journal. A bigger journal allows larger transactions which speed up write actions but might waste space. If you've got a second hard drive you could use an external journal but I've never done any benchmarking on that issue although I use it on my personal wannabe server (a raid1 and a single disk for the journal and unimportant data). I didn't comment on /var because I don't know how you use it. I suspect it to hold alot of temporal data like lock files, spools and so on. So there's a lot of creating and removing files going on, possibly in parallel. XFS is good in parallel and in creating files but terrible in removing files. Reiserfs with notail seems a good choice if you ask me (what you did ;) ) I'd also use ext2 on /usr/portage. These data don't need journaling. Everything's got an MD5-sum to make sure it's unchanged after a crash and you can easily resync. I found ext2 with 2k blocks to be faster than reiserfs3.6, even on read-performance. I've already made the partition as suggested in [1] I used this command: $ mke2fs -b 1024 -N 20 -m 0 -O dir_index I guess 1K block size would be faster?? I'm not sure. 2K blocks might reduce fragmentation. If you look at the output of find /usr/portage/ -type f | xargs du -h --apparent-size you'll see that there are quiet a few files larger than 1K but most are smaller and might stay that small. So yes, I think 1K is a good choice but you won't loose much with 2K, maybe you even gain some speed. If I were you, I'd also use separate volumes for /tmp and /var/tmp (without ccache) with xfs. What did you mean by 'without ccache'? I have ccache and I use it... I meant that you should keep ccache on a separate partition. I just think: Less stuff in the FS, less work on allocation and lookup, more speed. And there's a lot of stuff in 2GB ccache. By the way: I don't think /var/tmp is a good place for ccache (not technically, just for the sake of layout). I've moved it to /var/db since it's not really a bunch of temporary data but more like a changing database. /home could use data=journal. Those data are precious and if I remember correctly, this setting even brings an obscure (i.e. undocumented) speed improvement with many parallel disk accesses, for example in a multi-user environment. it's done, thanks, BTW what's your home partition FS? your choice is ext3 or reiserFS?? I use reiserfs3.6 without notail but that doesn't mean that it would be a good choice for you. I'm on laptop and disk space efficiency is a big topic for me so I use tail-packing wherever suitable. And yes, I am a fan of ReiserFS-3.6. I think it's the best multipurpose FS. You can easily adapt it for high performance or high disk space efficiency. If its journaling would be as good as Ext3's data=journal I'd use it everywhere except for small partitions (ext2) and big files (ext3 and xfs). One last thing, since I'm on LVM resizing the partition is a must feature, in ext3 I use resize2fs which works quite nicely, is resize_reiserfs as reliable as resize2fs is?? Yes, it's just as good and the sky's the limit for resizing :) Oh, by the way: If you choose to use XFS somewhere, keep in mind that you can't shrink and XFS-FS. Neither online nor offline. One last thing: It's a bit old but I think it's still interesting, especially for XFS-users: http://everything2.com/index.pl?node_id=1479435 signature.asc Description: This is a digitally signed message part
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
Wear leveling. Second UFD for occasional backup. Am I missing something, or does Portage only *write* to the database when you're [em,un]merging? If so, I don't see that there's much to worry about, even if you *are* running pure ~x86, and using overlays, like I am. The only real drawback I see is that UFDs are sufficiently stupid -- thanks, USB standards committee! -- that there's no way to interrogate them to get a report on sector wear. On Feb 15, 2008 5:06 PM, Neil Walker [EMAIL PROTECTED] wrote: Alois Hammer wrote: Suggestion: put your Portage and database trees on flash storage. There is no way I would do that or recommend it to anyone. Those devices have a very, very short life if written to frequently. Portage isn't a big problem because an emerge --sync will restore it - but database trees? You have to be kidding. Be lucky, Neil -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- gentoo-user@lists.gentoo.org mailing list -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
I personally prefer JFS to XFS and have used it for years on my servers and laptop with no problem other than hardware errors (and if the hardware fails the fs will not help you). I had system board problems in the laptop and a bad RAID controller in the server this last year :(. Other than that I always recovered from outages with a journal-replay. Your plan looks rational except I wouldn't use ext3 for storing video - it's slow when deleting large files and large numbers of files - xfs or jfs would be better. For that matter you may care to split your video and mp3 storage because mp3's are small and videos are usually large. If your videos are generally uniform (like on mythtv, allocated in half-hour multiples) storing them all in a filesystem of their own will reduce fragmentation. If you're doing write-once it doesn't matter so much, but if you delete things a lot it's more important to performance. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Ext4 status - Alternative to ext2/3 for gentoo portage and more
This One Time, at Band Camp, Florian Philipp [EMAIL PROTECTED] said, On Sat, Feb 16, 2008 at 01:50:04AM +0100: On Sat, 2008-02-16 at 00:32 +0100, Wael Nasreddine wrote: To your filesystem scheme: Why do you use xfs for usr? AFAIK XFS is good at write speed but not worth the trouble when reading data and data in usr is usually written once, updated every few months and read many times a week (on rebooting Desktop PCs maybe once a day). I'd use reiserfs3.6, maybe even without notail to make it more space efficient. I don't use XFS, curently I only have / and /home and I want to split it to more smaller partitions, I'm on LVM so it's easy, anyway I'm going with ReiserFS for /usr /var, would you please suggest mkfs.reiserfs options as I have nerver used ReiserFS-3 before (yep 5 years using linux and I've always used ext3...) also You didn't mention /var, would you say ReiserFS-3 is a good choice as well? I don't think there's alot to do when creating a reiserfs. You could change the number of blocks for the journal. A bigger journal allows larger transactions which speed up write actions but might waste space. If you've got a second hard drive you could use an external journal but I've never done any benchmarking on that issue although I use it on my personal wannabe server (a raid1 and a single disk for the journal and unimportant data). I didn't comment on /var because I don't know how you use it. I suspect it to hold alot of temporal data like lock files, spools and so on. So there's a lot of creating and removing files going on, possibly in parallel. XFS is good in parallel and in creating files but terrible in removing files. Reiserfs with notail seems a good choice if you ask me (what you did ;) ) I'd also use ext2 on /usr/portage. These data don't need journaling. Everything's got an MD5-sum to make sure it's unchanged after a crash and you can easily resync. I found ext2 with 2k blocks to be faster than reiserfs3.6, even on read-performance. I've already made the partition as suggested in [1] I used this command: $ mke2fs -b 1024 -N 20 -m 0 -O dir_index I guess 1K block size would be faster?? I'm not sure. 2K blocks might reduce fragmentation. If you look at the output of find /usr/portage/ -type f | xargs du -h --apparent-size you'll see that there are quiet a few files larger than 1K but most are smaller and might stay that small. So yes, I think 1K is a good choice but you won't loose much with 2K, maybe you even gain some speed. If I were you, I'd also use separate volumes for /tmp and /var/tmp (without ccache) with xfs. What did you mean by 'without ccache'? I have ccache and I use it... I meant that you should keep ccache on a separate partition. I just think: Less stuff in the FS, less work on allocation and lookup, more speed. And there's a lot of stuff in 2GB ccache. By the way: I don't think /var/tmp is a good place for ccache (not technically, just for the sake of layout). I've moved it to /var/db since it's not really a bunch of temporary data but more like a changing database. /home could use data=journal. Those data are precious and if I remember correctly, this setting even brings an obscure (i.e. undocumented) speed improvement with many parallel disk accesses, for example in a multi-user environment. it's done, thanks, BTW what's your home partition FS? your choice is ext3 or reiserFS?? I use reiserfs3.6 without notail but that doesn't mean that it would be a good choice for you. I'm on laptop and disk space efficiency is a big topic for me so I use tail-packing wherever suitable. And yes, I am a fan of ReiserFS-3.6. I think it's the best multipurpose FS. You can easily adapt it for high performance or high disk space efficiency. If its journaling would be as good as Ext3's data=journal I'd use it everywhere except for small partitions (ext2) and big files (ext3 and xfs). One last thing, since I'm on LVM resizing the partition is a must feature, in ext3 I use resize2fs which works quite nicely, is resize_reiserfs as reliable as resize2fs is?? Yes, it's just as good and the sky's the limit for resizing :) Oh, by the way: If you choose to use XFS somewhere, keep in mind that you can't shrink and XFS-FS. Neither online nor offline. One last thing: It's a bit old but I think it's still interesting, especially for XFS-users: http://everything2.com/index.pl?node_id=1479435 Thank you for your detailed answer it helped a lot, I just finished resizing/migrating all partitions, Though I still have the Storage partition, which is for my Mp3z and is almost 70Gb, with ext3, I'll see later if I do migrate to ReiserFS or not but the rest is done, please take a look at the file attached... and if you have any more suggestions please do tell me. Thanks a lot guys -- Wael Nasreddine http://wael.nasreddine.com PGP: 1024D/C8DD18A2 06F6 1622 4BC8
