On 12 May 2015 at 16:01, William A. Mahaffey III <w...@hiwaay.net> wrote: > On 05/12/15 02:32, David Brownlee wrote: >> >> On 11 May 2015 at 23:46, William A. Mahaffey III <w...@hiwaay.net> wrote: >> >> If you are using RAID5 I would strongly recommend keeping to >> "power-of-two + 1" components, to keep the stripe size as a nice power >> of two, otherwise performance is... significantly impaired. > > Hmmmm .... Could you amplify on that point a bit ? I am intending to > maximize available storage & have already procured the mbd & 6 drives, but I > could rethink things if my possibly hasty choices would be too burdensome
For RAID5 to perform efficiently data should be written in units which re aligned with the RAID stripes and are a multiple of stripe in size, otherwise a simple write changes into a read of stripe, modification of the affected part and then a write. Filesystems tend to have sectors and blocks which are powers of two, so the easiest way to arrange this for ffs is for the filesystem block size to be a multiple of the stripe size ("1" is a fine multiple in this case). This is similar to the issue with sector drives which have 4K sectors but present them 512byte sectors - if a filesystem is not 4K aligned then write performance suffers horribly. >> If you do not need to maximise the space you will always get better >> performance from RAID1 (or RAID10). For that I would RAID1 the disks >> in pairs, then RAID0 two of them to give a 1TB and fast 2TB storage >> units, which would them be partitioned up as needed. It also makes it >> simpler to later replace the pair or the four disks while leaving the >> other set. > > I do want to maximize storage space, will go RAID5 almost certainly for my > largest storage pool, /home. >> As long as you are below the 2TB limit for any given component you can >> use disklabels which are much simpler than gpt with wedges. NetBSD is >> moving more to wedges, but netbsd-6 is probably not the version to do >> it on by choice :) On that note I would probably put a netbsd-7 BETA >> on the box and just update when the full release comes out. > > I want the (possibly perceived) reliability of the 6.1.5 over anything > called BETA :-/ .... NetBSD is very conservative about shipping releases. I think the BETA is very close to what will be released and I've found it very stable. >> If you want to maximise space with some redundancy then as you say, >> RAID5 is the way to go for the bulk of the storage. >> >> A while back I setup a machine with 5 * 2TB disks with netbsd-6, with >> small RAID1 partitions for root and the bulk as RAID5 >> http://abs0d.blogspot.co.uk/2011/08/setting-up-8tb-netbsd-file-server.html >> (wow, was that really four years ago) - in your position I might keep >> one 1TB as a scratch/build space and then RAID up the rest. >> >> If you have time definitely experiment, get a feel for the different >> performance available from the different options. > > *Wow*, another fabulous resource. Your blog documents almost verbatim what I > have in mind. I am going w/ 6 drives (already procured, 6 SATA3 slots on the > mbd, done deal), but philosophically very close to what you describe. 1 > question: if you were doing this again today, would it be fdisk or GPT ? If I had >2TB drives it would be TB :) If not, I would still stick with fdisk. The complexity of gpt setup and wedge autoconfiguration is still greater than fdisk and disklabel. I know I'm going to have to move to it at some point, but I'm going to hold off until I need to > I think I am looking at 4 partitions per drive, ~16 GB for / (RAID1, 2 drives) > & /usr (4 drives, RAID10), 16 GB for swap (kernel driver, all 6 drives), 16 > - 32 GB for /var (RAID5, all 6 drives), & the rest for /home (RAID5, all 6 > drives). TIA & thanks again. I would definitely hold off on RAID5 for everything except the large /home. RAID1 is much simpler and more performance for writes. I would also try to avoid configuring multiple RAID5s across overlapping sets of disks, while it theoretically provides more IO bandwidth, that bandwidth will be having to compete with all the other filesystems and swap usage on the system. If you wanted to use all six disks: - 32G(RAID1 root+usr) 910G(non raid scratch space) - 32G(RAID1 root+usr) 910G(RAID5 home) - 32G(RAID1 var) 910G(RAID5 home) - 32G(RAID1 var) 910G(RAID5 home) - 32G(RAID1 swap+spare) 910G(RAID5 home) - 32G(RAID1 swap+spare) 910G(RAID5 home) 32GB space notes: - This gives you three 32GB RAID1 'pools' to allocate everything outside of /home - Can adjust the 32G up or down before partitioning, but all should be the same - In the suggestion, root+usr are kept on the same RAID (and could be a single partition), so that the system can have all of the userland available with only one disk attached, and a 'spare' partition is left in case of later moderate additional space needs - maybe an extra partition for /usr/pkg?, or for /var/pgsql, etc - Obviously allocate usage within pools to taste - could put /usr on a separate raid to provide more IO bandwidth for root & usr 910GB space notes: - This gives 5* 910GB RAID5, which provides 4*910G (or 3640G) of space - One disk is not included in the RAID5. This could be saved as a spare for a RAID5 component failure (though a better approach might be to have a disk on the desk next to the machine :), or used as non raided scratch space. If it will not be active, then probably best to put it on one of the components for the heaviest used 32G, or the most important 32G Note in the above that IO to /home will hit (almost) all disks, and will affect all of the 32GB pools, so if you have heavy IO to /home do not expect blistering performance from any filesystem. On the other hand when /home has very light IO then you should have relatively nice multi spindle performance from the other filesystems. Having said all that, if I had the time to play I would install onto a USB key, then script up the building and partitioning of the system in many different forms and then chroot into the result and run some tests to see how it performs.