Others will know better, but I always thought a Raid5 with four disks spends most time with read-modify-write cycles as usually no requests will fit the stripe size.
Did you try ZFS/raidz on NetBSD? -- J. Hannken-Illjes - [email protected] > On 20. Jun 2026, at 18:55, smurfd <[email protected]> wrote: > > Hey, > > This will be a long one, sorry for this. > I have created a RAID5 NAS using NetBSD 10.1, and im experiencing slow > writing speeds. Or is my expectations all wrong? :) > I earlier had FreeBSD and ZFS zraid on the same Ugreen DXP4800 plus > (https://nas-eu.ugreen.com/products/ugreen-nasync-dxp4800-plus-nas-storage) > The NAS drives are 4x7TB, less than a year old Segate Ironwolf NAS 3.5". > NetBSD is installed first on the SSD, using a base install. Then the Raid is > created manually... > > Have asked here: > https://www.unitedbsd.com/d/1646-raid5-slow-write-on-big-disks/8 > Just now redid the raid, for the Xth time, to compare the speeds i got using > FreeBSD (because i didnt remember/had that written down) > I know, comparing zraid vs raid5 is like comparing apples to oranges, > right?... > > What write speeds should i expect?! is probably the most important question... > Or where did i go wrong, because user error is VERY likely. > > First, i tried to follow the Summary section in the > https://man.netbsd.org/raidctl.8, realizing that Disklabel supports no larger > than 2TB disks. > So found this: https://wiki.netbsd.org/users/mlelstv/using-large-disks/ and > tried to follow the Raidframe section. > Then i got the correct size i expected... > > The speed though, is slow! (again compared to zfs zraid) > When i had FreeBSD installed, i did the following. Installed the system to > the SSD. > Then, did this, where **** is repeaded 4 times for the disks... > # gpart destroy -F ada0 **** > # gpart create -s gpt ada0 **** > # gpart add -t freebsd-zfs ada0 **** > # zpool create -f island raidz /dev/ada0p1 /dev/ada1p1 /dev/ada2p1 > /dev/ada3p1 (this was instant) > island/scumm 21T 140K 21T 0% /zstorage/scumm > > This is the speed... > # dd if=/dev/zero bs=1024k count=1000 of=/zstorage/scumm/test.txt > 1048576000 bytes transferred in 0.259733 secs (4037123661 bytes/sec) > # dd if=/dev/zero bs=4096k count=1000 of=/zstorage/scumm/test.txt > 4194304000 bytes transferred in 1.534966 secs (2732505282 bytes/sec) > # dd if=/dev/zero bs=1024 count=1000 of=/zstorage/scumm/test.txt > 1024000 bytes transferred in 0.004667 secs (219403356 bytes/sec) > > Then on NetBSD, installed on the SSD i did this: > > # gpt destroy wd0 **** > # gpt create -Af wd0 **** > # gpt add -t raid -l raid5@wd0 -b $(( 2048 )) -s 15628051053 wd0 **** > > # raidctl -C /etc/raid5.conf raid5 > # raidctl -I 13371337 raid5 > # raidctl -iv raid5 (This took around 72+ hours) > # gpt create -Af raid5 > # gpt add -a 1024 -t ffs -l island raid5 > /dev/rraid5: Partition 1 added: 49f48d5a-b10e-11dc-b99b-0019d1879648 34 > 46884152860 > # dkctl raid5 addwedge island 34 46884152860 ffs > # newfs -O2 -b 65536 -s -1g /dev/dk7 (trying to enable > https://man.netbsd.org/wapbl.4) > # tunefs -l 1g /dev/dk7 > # mount -o log /dev/dk7 /mnt > # umount /mnt > # mount /dev/dk7 /mnt/island/ > /dev/dk7 22T 8.0K 21T 0% /mnt/island > > from /etc/fstab > /dev/dk7 /mnt/island ffs rw,noatime,log 1 1 > > This one, was done before initializing the raid, on one disk > nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/test.txt > 1048576000 bytes transferred in 2.201 secs (476408905 bytes/sec) > > Writing to the raid: > nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/island/test.txt > 1048576000 bytes transferred in 68.984 secs (15200278 bytes/sec) > nas1# dd if=/dev/zero bs=4096k count=1000 of=/mnt/island/test.txt > 4194304000 bytes transferred in 478.951 secs (8757271 bytes/sec) > nas1# dd if=/dev/zero bs=1024 count=1000 of=/mnt/island/test.txt > 1024000 bytes transferred in 0.054 secs (18962962 bytes/sec) > > If i tried bs=512k and count=1000, i get around 1.6Billion bytes per sek > write, woohoo... > Then try the same with count=2000 and get 1.3 million bytes per sek write > hmmm .... > > This is /etc/raid5.conf > START array > # numRow numCol numSpare > 1 4 0 > > START disks > /dev/dk3 > /dev/dk4 > /dev/dk5 > /dev/dk6 > > START layout > # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_5 > 32 1 1 5 > > START queue > fifo 100 > > > Summary: > It seems like this is 200 times slower than FreeBSD zraid, is that > reasonable? or where did i go wrong?! > Again, only curious. Was thinking about installing OpenBSD, they have a more > similar raid system compared to netbsd(more or less exactly the same:)) > > Could it be worth adding a Big disk section to the raidctl manpage: > https://man.netbsd.org/raidctl.8 > > Thanks in advance! > Br Nicklas >
