[gentoo-user] RAID6/ext4 for /?
I haven't seen much discussion of ext4. Are people using it? I have a new RAID6 I'm considering putting it on. I'd like to possibly make the partition /, copy the existing system there and then use an initrd to boot when I finally figure that out. (Never used one...) Anything specific about ext4 that would make this a problem vs ext3. Cheers, Mark
Re: [gentoo-user] RAID6/ext4 for /?
On 31/12/2010, at 4:16pm, Mark Knecht wrote: I haven't seen much discussion of ext4. Are people using it? Yeah, it seems pretty good. Anecdotally it seems stable enough, I don't see why it should be less so than ext3, now. Deletes are *much* faster than with ext3. Stroller.
Re: [gentoo-user] RAID6/ext4 for /?
On Fri, Dec 31, 2010 at 8:41 AM, Stroller strol...@stellar.eclipse.co.uk wrote: On 31/12/2010, at 4:16pm, Mark Knecht wrote: I haven't seen much discussion of ext4. Are people using it? Yeah, it seems pretty good. Anecdotally it seems stable enough, I don't see why it should be less so than ext3, now. Deletes are *much* faster than with ext3. Stroller. Thanks Stroller. I saw a web page where Google announced this December that they were going to be using it. I figured I might as well finally get on board. The next thing I need to understand is the initramfs stuff. My existing RAID1 (md5 below, 3 disks) uses a 0.90 Super Block which get auto-assembled by the kernel at boot. The new RAID6 (md3 below, 5 disks) uses Ver 1.2 which, as I understand it, won't get auto-assembled and requires the initramfs. I'm currently going through the Gentoo docs to learn about setting that up. Cheers, Mark m...@c2stable ~ $ cat /proc/mdstat Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] md6 : active raid1 sdc6[2] sdb6[1] sda6[0] 247416933 blocks super 1.1 [3/3] [UUU] md3 : active raid6 sdb3[1] sdc3[2] sda3[0] sdd3[3] sde3[4] 157305168 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5] [U] md5 : active raid1 sdc5[2] sdb5[1] sda5[0] 52436032 blocks [3/3] [UUU] unused devices: none m...@c2stable ~ $
Re: [gentoo-user] RAID6/ext4 for /?
Apparently, though unproven, at 18:41 on Friday 31 December 2010, Stroller did opine thusly: On 31/12/2010, at 4:16pm, Mark Knecht wrote: I haven't seen much discussion of ext4. Are people using it? Yeah, it seems pretty good. Anecdotally it seems stable enough, I don't see why it should be less so than ext3, now. Deletes are *much* faster than with ext3. I get the same with ext4 on three machines: Latest Ubuntu on the Acer notebook This here gentoo notebook Android Donut on the phone Seems reliably enough - no problems yet after 1 full year. Nothing unusual about my usage except the notebook compiles stuff almost constantly, is always running three VMs at least and is hardly never switched off -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] RAID6/ext4 for /?
On Fri, Dec 31, 2010 at 12:53 PM, Alan McKinnon alan.mckin...@gmail.com wrote: Apparently, though unproven, at 18:41 on Friday 31 December 2010, Stroller did opine thusly: On 31/12/2010, at 4:16pm, Mark Knecht wrote: I haven't seen much discussion of ext4. Are people using it? Yeah, it seems pretty good. Anecdotally it seems stable enough, I don't see why it should be less so than ext3, now. Deletes are *much* faster than with ext3. I get the same with ext4 on three machines: Latest Ubuntu on the Acer notebook This here gentoo notebook Android Donut on the phone Seems reliably enough - no problems yet after 1 full year. Nothing unusual about my usage except the notebook compiles stuff almost constantly, is always running three VMs at least and is hardly never switched off -- alan dot mckinnon at gmail dot com Thanks Alan. That's good info. As this is intended to be more or less a duplicate of the existing system on this box, just placed on a new RAID6, I've opted not to do the disk copy as per the thread this week. No LiveCD, I just followed the Gentoo install from within a terminal on this box. It's flying along with nearly everything done expect the new kernel. The holdup there will be figuring out what an initramfs really is and what really needs to be included in it. I'm following this: http://en.gentoo-wiki.com/wiki/Initramfs which is reasonable, We'll see how it goes. I will say that the install on this 5-disk RAID6 running ext4 seems speedy compared to the 3-disk RAID1 running ext3 that I'm currently using. Probably I'm just imagining that... Cheers, Mark
Re: [gentoo-user] RAID6/ext4 for /?
Apparently, though unproven, at 23:18 on Friday 31 December 2010, Mark Knecht did opine thusly: On Fri, Dec 31, 2010 at 12:53 PM, Alan McKinnon alan.mckin...@gmail.com wrote: Apparently, though unproven, at 18:41 on Friday 31 December 2010, Stroller did opine thusly: On 31/12/2010, at 4:16pm, Mark Knecht wrote: I haven't seen much discussion of ext4. Are people using it? Yeah, it seems pretty good. Anecdotally it seems stable enough, I don't see why it should be less so than ext3, now. Deletes are *much* faster than with ext3. I get the same with ext4 on three machines: Latest Ubuntu on the Acer notebook This here gentoo notebook Android Donut on the phone Seems reliably enough - no problems yet after 1 full year. Nothing unusual about my usage except the notebook compiles stuff almost constantly, is always running three VMs at least and is hardly never switched off -- alan dot mckinnon at gmail dot com Thanks Alan. That's good info. As this is intended to be more or less a duplicate of the existing system on this box, just placed on a new RAID6, I've opted not to do the disk copy as per the thread this week. No LiveCD, I just followed the Gentoo install from within a terminal on this box. It's flying along with nearly everything done expect the new kernel. The holdup there will be figuring out what an initramfs really is and what really needs to be included in it. I'm following this: http://en.gentoo-wiki.com/wiki/Initramfs which is reasonable, We'll see how it goes. I will say that the install on this 5-disk RAID6 running ext4 seems speedy compared to the 3-disk RAID1 running ext3 that I'm currently using. Probably I'm just imagining that... Maybe you do have observer bias. Or maybe you bought new shiny *faster* disks :-) -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] RAID6/ext4 for /?
On Fri, Dec 31, 2010 at 1:53 PM, Alan McKinnon alan.mckin...@gmail.com wrote: Apparently, though unproven, at 23:18 on Friday 31 December 2010, Mark Knecht SNIP I will say that the install on this 5-disk RAID6 running ext4 seems speedy compared to the 3-disk RAID1 running ext3 that I'm currently using. Probably I'm just imagining that... Maybe you do have observer bias. Or maybe you bought new shiny *faster* disks :-) I vote for the former. Same disks, just different partitions... - Mark
Re: [gentoo-user] RAID6 for / ?
Am 2010-12-11 00:08, schrieb Neil Bothwick: On Fri, 10 Dec 2010 14:47:26 +0100, Stefan G. Weichinger wrote: Since / only needs about 200MB, space isn't really an issue, and RAID1 gives the highest redundancy. The main reason I don't put / on a higher RAID level is that it adds the complication of another filesystem and partition on each drive. / + one LVM keeps it nice and simple. OK, I see your point. My / is usually bigger, it seems that you use other/more separate partitions/LVs for stuff like /var etc. Of course. If you're going to use LVM, you may as well put everything on it. Set it up today. small RAID1 for boot (6 devices) 32 gigs RAID6 for / (right now it's all in there... /var etc ... just to keep it simple for the start) the rest (aside from swap) went into a LVM-VG (sorry, folks) I will learn about bind-mounts on my testboxes, for the customer I decided to stay with what I know so far. And, yes, I took care of chunks, stride, and stripes ;-) Thanks all .. Stefan
Re: [gentoo-user] RAID6 for / ?
Am 09.12.2010 18:14, schrieb Neil Bothwick: On Thu, 09 Dec 2010 17:08:42 +0100, Stefan G. Weichinger wrote: Wouldn't it be even more efficient (in terms of not wasting space) to create that 6-devices-RAID1 smaller, for /boot and a second array, with RAID6, for / ? Less space wasted, higher redundancy for / ... ? Since / only needs about 200MB, space isn't really an issue, and RAID1 gives the highest redundancy. The main reason I don't put / on a higher RAID level is that it adds the complication of another filesystem and partition on each drive. / + one LVM keeps it nice and simple. OK, I see your point. My / is usually bigger, it seems that you use other/more separate partitions/LVs for stuff like /var etc. Thanks for that, Stefan
Re: [gentoo-user] RAID6 for / ?
Am 09.12.2010 18:21, schrieb Volker Armin Hemmann: Put /boot on raid1, / on raid6. Don't bother with lvm - it is just another layer that can go wrong. You mean don't use lvm for / ? ... for other stuff it's very useful, isn't it? ;-) I never put / on lvm, yep. If Raid6 is like raid5 you should be able to have the kernel auto assemble everything, so no initrd is needed. Pay attention to the metadata format when creating the raid. Also have a look at stripe sizes. And stripe_cache_size. Thanks for your reminders. What exactly do you think of with stripes and sizes? You point at the performance-impact? large files vs. small files etc? Thanks, Stefan
Re: [gentoo-user] RAID6 for / ?
On Friday 10 December 2010, Stefan G. Weichinger wrote: Am 09.12.2010 18:21, schrieb Volker Armin Hemmann: Put /boot on raid1, / on raid6. Don't bother with lvm - it is just another layer that can go wrong. You mean don't use lvm for / ? ... for other stuff it's very useful, isn't it? no, I mean don't use lvm. It is a very compley, easily broken layer reducing data safety. There is no need for lvm with bind mounting and ln -s. If Raid6 is like raid5 you should be able to have the kernel auto assemble everything, so no initrd is needed. Pay attention to the metadata format when creating the raid. Also have a look at stripe sizes. And stripe_cache_size. Thanks for your reminders. What exactly do you think of with stripes and sizes? You point at the performance-impact? large files vs. small files etc? stripe size has nothing to do with big and small files. But choosing the wrong stripe size can impact your performance very, very badly. We are talking about abysmal performance, Challenger depht abysmal. XFS and ext4 - for both is a lot of documentation available about choosing the right stripe size. stripe_cache_size can be set in /sys and has a 4-5x performance impact on raid5 (where I tried it). A good size for me is 8192. For example.
Re: [gentoo-user] RAID6 for / ?
Am 2010-12-10 16:41, schrieb Volker Armin Hemmann: On Friday 10 December 2010, Stefan G. Weichinger wrote: Am 09.12.2010 18:21, schrieb Volker Armin Hemmann: Put /boot on raid1, / on raid6. Don't bother with lvm - it is just another layer that can go wrong. You mean don't use lvm for / ? ... for other stuff it's very useful, isn't it? no, I mean don't use lvm. It is a very compley, easily broken layer reducing data safety. ok hmmm There is no need for lvm with bind mounting and ln -s. I don't fully get that yet ... If Raid6 is like raid5 you should be able to have the kernel auto assemble everything, so no initrd is needed. Pay attention to the metadata format when creating the raid. Also have a look at stripe sizes. And stripe_cache_size. Thanks for your reminders. What exactly do you think of with stripes and sizes? You point at the performance-impact? large files vs. small files etc? stripe size has nothing to do with big and small files. But choosing the wrong stripe size can impact your performance very, very badly. We are talking about abysmal performance, Challenger depht abysmal. XFS and ext4 - for both is a lot of documentation available about choosing the right stripe size. stripe_cache_size can be set in /sys and has a 4-5x performance impact on raid5 (where I tried it). A good size for me is 8192. For example. Ah, yes, already found that and I will check that out soon. Thanks!
Re: [gentoo-user] RAID6 for / ?
On Friday 10 December 2010, Stefan G. Weichinger wrote: Am 2010-12-10 16:41, schrieb Volker Armin Hemmann: On Friday 10 December 2010, Stefan G. Weichinger wrote: Am 09.12.2010 18:21, schrieb Volker Armin Hemmann: Put /boot on raid1, / on raid6. Don't bother with lvm - it is just another layer that can go wrong. You mean don't use lvm for / ? ... for other stuff it's very useful, isn't it? no, I mean don't use lvm. It is a very compley, easily broken layer reducing data safety. ok hmmm There is no need for lvm with bind mounting and ln -s. I don't fully get that yet ... well, what is so 'great' about lvm? That you can shove around free space where it is needed. You can do the same with bind mounting. With the additional bonus that mount simply works. Unlike lvm. Btw, does lvm still eat barriers?
Re: [gentoo-user] RAID6 for / ?
Am 2010-12-10 19:03, schrieb Volker Armin Hemmann: well, what is so 'great' about lvm? That you can shove around free space where it is needed. You can do the same with bind mounting. With the additional bonus that mount simply works. Unlike lvm. Btw, does lvm still eat barriers? dunno ;-) I am just used to LVM and it so far just works for me. but I am always learning ...
Re: [gentoo-user] RAID6 for / ?
On Fri, 10 Dec 2010 14:47:26 +0100, Stefan G. Weichinger wrote: Since / only needs about 200MB, space isn't really an issue, and RAID1 gives the highest redundancy. The main reason I don't put / on a higher RAID level is that it adds the complication of another filesystem and partition on each drive. / + one LVM keeps it nice and simple. OK, I see your point. My / is usually bigger, it seems that you use other/more separate partitions/LVs for stuff like /var etc. Of course. If you're going to use LVM, you may as well put everything on it. -- Neil Bothwick With 7 billion people on earth chances are slim it will ever be *your* day. signature.asc Description: PGP signature
[gentoo-user] RAID6 for / ?
I have to install a new amd64-gentoo server next week. Business as usual. I now see that the box has 6 SATA-hdds with 320 GB each. A software-RAID is planned, RAID6 for the data (in a LVM2-VG) ... I usually set up /boot on a RAID1 on the first two disks and often also / on simply a mirror. Would you guy recommend to use RAID6 for / as well? Thoughts welcome! Thanks, Stefan
Re: [gentoo-user] RAID6 for / ?
On Thu, 09 Dec 2010 12:14:44 +0100, Stefan G. Weichinger wrote: A software-RAID is planned, RAID6 for the data (in a LVM2-VG) ... I usually set up /boot on a RAID1 on the first two disks and often also / on simply a mirror. Would you guy recommend to use RAID6 for / as well? I don't bother with a separate /boot, especially when / is separate anyway. I'd use / (including /boot) as RAID1 on all the disks, then the RAID6 LVM for everything else. Otherwise you have to mess around with initrds or scripts to mount / from the LVM. -- Neil Bothwick Is that woof feed me; woof walk me; woof there's a burglar? What?? signature.asc Description: PGP signature
Re: [gentoo-user] RAID6 for / ?
Am 09.12.2010 15:14, schrieb Neil Bothwick: I don't bother with a separate /boot, especially when / is separate anyway. I'd use / (including /boot) as RAID1 on all the disks, then the RAID6 LVM for everything else. Otherwise you have to mess around with initrds or scripts to mount / from the LVM. I never did RAID1 with more than 2 disks, you suggest something like: mdadm --create /dev/md0 --level=1 --raid-disks=6 /dev/sd[abcdef]1 ? This would result in /dev/md0 having a usable size of 3 disks? Thanks, Stefan
Re: [gentoo-user] RAID6 for / ?
On Thu, 09 Dec 2010 15:53:24 +0100, Stefan G. Weichinger wrote: I never did RAID1 with more than 2 disks, you suggest something like: mdadm --create /dev/md0 --level=1 --raid-disks=6 /dev/sd[abcdef]1 Yes This would result in /dev/md0 having a usable size of 3 disks? No, the size is that of one disk. The reason for using all disks is that otherwise the space would go unused and this way you can boot from any of the disks in the event of failures. I've never tried it with 6, but my desktop is set up like this with a three disk RAID1 for / and LVM on RAID5 for everything else. -- Neil Bothwick Beware of cover disks bearing upgrades. signature.asc Description: PGP signature
Re: [gentoo-user] RAID6 for / ?
Am 09.12.2010 16:44, schrieb Neil Bothwick: On Thu, 09 Dec 2010 15:53:24 +0100, Stefan G. Weichinger wrote: I never did RAID1 with more than 2 disks, you suggest something like: mdadm --create /dev/md0 --level=1 --raid-disks=6 /dev/sd[abcdef]1 Yes fine This would result in /dev/md0 having a usable size of 3 disks? No, the size is that of one disk. The reason for using all disks is that otherwise the space would go unused and this way you can boot from any of the disks in the event of failures. Sure, understood. Didn't really know about the resulting size ... interesting, but logical ;) I've never tried it with 6, but my desktop is set up like this with a three disk RAID1 for / and LVM on RAID5 for everything else. So I would have to install GRUB in all 6 MBRs as well, sure ... Wouldn't it be even more efficient (in terms of not wasting space) to create that 6-devices-RAID1 smaller, for /boot and a second array, with RAID6, for / ? Less space wasted, higher redundancy for / ... ? Sounds good to me right now (I will see if it changes after hitting SEND). Stefan
Re: [gentoo-user] RAID6 for / ?
On Thu, 09 Dec 2010 17:08:42 +0100, Stefan G. Weichinger wrote: I've never tried it with 6, but my desktop is set up like this with a three disk RAID1 for / and LVM on RAID5 for everything else. So I would have to install GRUB in all 6 MBRs as well, sure ... Wouldn't it be even more efficient (in terms of not wasting space) to create that 6-devices-RAID1 smaller, for /boot and a second array, with RAID6, for / ? Less space wasted, higher redundancy for / ... ? Since / only needs about 200MB, space isn't really an issue, and RAID1 gives the highest redundancy. The main reason I don't put / on a higher RAID level is that it adds the complication of another filesystem and partition on each drive. / + one LVM keeps it nice and simple. -- Neil Bothwick .-Stealth Tagline signature.asc Description: PGP signature
Re: [gentoo-user] RAID6 for / ?
Put /boot on raid1, / on raid6. Don't bother with lvm - it is just another layer that can go wrong. If Raid6 is like raid5 you should be able to have the kernel auto assemble everything, so no initrd is needed. Pay attention to the metadata format when creating the raid. Also have a look at stripe sizes. And stripe_cache_size.
