Hello,
To answer your most important question first: Are your expectations wrong?
Yes, slightly.
When you tested ZFS on FreeBSD using dd if=/dev/zero, you weren't actually
testing your hard drives. You were testing your RAM and CPU.
* ZFS is a highly intelligent, Copy-on-Write filesystem. By default, it
uses compression. When you feed it a stream of zeroes, ZFS compresses that
data into virtually nothing and caches it in RAM (the ARC).
* The 4.03 GB/sec write speed you saw is physically impossible for four
Seagate IronWolf NAS drives. Those drives max out at around 210–250 MB/s
each. Even in a perfect RAID0 (no parity), the theoretical maximum physical
write speed would be around 1 GB/s.
What should you expect?On a perfectly tuned 4-disk RAID5 array of modern
7TB spinning drives, you should expect sustained sequential write speeds
somewhere between 300 MB/s and 500 MB/s.
Your NetBSD speeds are genuinely terrible, and there are two main culprits
working together to sabotage your performance: Stripe Misalignmentand the
Read-Modify-Write (RMW) penalty.
When you configure RAID5, data is striped across the disks in chunks, with
one block reserved for parity. If you write a file that is smaller than the
stripe size, or misaligned with the stripe, the RAID controller (or
RAIDframe in this case) cannot just write the data. It has to:
1. Read the old data and the old parity from the disks.
2. Modify the data in memory.
3. Write the new data and the new parity back to the disks.
This turns one write operation into four separate I/O operations.
In your /etc/raid5.conf, you defined:
sectPerSU 32
RAIDframe calculates this in 512-byte sectors.
32 \times 512 \text{ bytes} = 16\text{ KB} per disk.
Because you have 4 disks in RAID5, 3 are used for data and 1 for parity.
Your total data stripe width is:
3 \text{ data disks} \times 16\text{ KB} = 48\text{ KB stripe width}
However, when you created your filesystem, you used:
newfs -b 65536 (which dictates a 64 KB block size).
Because your filesystem blocks (64 KB) don't match your RAID stripes (48
KB), every single write you make overlaps multiple stripes. This
guarantees a massive Read-Modify-Write penalty for every single block of
data. The initial 1.6 GB/s burst you saw was NetBSD writing to your system
RAM (buffer cache). The 1.3 MB/s drop happened when the RAM filled up and
the system desperately tried to flush those misaligned blocks to the
physical disks.
To fix this, we need to ensure your filesystem block size fits neatly into
your RAID stripe units.
Since you have 4 disks (3 data), it is mathematically impossible to make
the total stripe width a perfect power-of-two (like 64KB) because 3 is not
a power-of-two. The best workaround is to make the *individual disk stripe
unit* match your filesystem block size.
Change your /etc/raid5.conf to use
128 sectors per SU:
START array
1 4 0
START disks
/dev/dk3
/dev/dk4
/dev/dk5
/dev/dk6
START layout
# sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_5
128 1 1 5
START queue
fifo 256
* Why 128? 128 \times 512 \text{ bytes} = 64\text{ KB}.
* Now, a 64 KB filesystem block from FFS will fit perfectly onto a
*single* disk's chunk. RAIDframe will only need to calculate parity against
that specific chunk, rather than spanning multiple chunks and fragmenting
the I/O.
* I also bumped your queue size to 256 to give the disks a bit more
breathing room for optimization.
You enabled WAPBL using tunefs -l 1g. WAPBL is great for crash consistency,
but it works by writing small, synchronous metadata updates to the disk. On
RAID5, small synchronous writes trigger the Read-Modify-Write penalty
constantly.
Once you rebuild the RAID with sectPerSU 128, test the write speeds before
enabling the WAPBL log. You'll likely see a massive improvement. If
enabling the log slows things down again, consider placing the WAPBL log on
your SSD instead of the RAID array.
You aren't crazy, and NetBSD isn't inherently 200x slower. ZFS was just
tricking you with RAM speeds, and your NetBSD array was mathematically
fighting against itself due to a stripe-size mismatch. And yes, the raidctl
man page absolutely needs a modern "large disk" section!
Regards,
Arya
On Sat, 20 Jun, 2026, 10:26 pm smurfd, <[email protected]> wrote:
> Hey,
>
> This will be a long one, sorry for this.
> I have created a RAID5 NAS using NetBSD 10.1, and im experiencing slow
> writing speeds. Or is my expectations all wrong? :)
> I earlier had FreeBSD and ZFS zraid on the same Ugreen DXP4800 plus
> (https://nas-eu.ugreen.com/products/ugreen-nasync-dxp4800-plus-nas-storage
> )
> The NAS drives are 4x7TB, less than a year old Segate Ironwolf NAS 3.5".
> NetBSD is installed first on the SSD, using a base install. Then the
> Raid is created manually...
>
> Have asked here:
> https://www.unitedbsd.com/d/1646-raid5-slow-write-on-big-disks/8
> Just now redid the raid, for the Xth time, to compare the speeds i got
> using FreeBSD (because i didnt remember/had that written down)
> I know, comparing zraid vs raid5 is like comparing apples to oranges,
> right?...
>
> What write speeds should i expect?! is probably the most important
> question...
> Or where did i go wrong, because user error is VERY likely.
>
> First, i tried to follow the Summary section in the
> https://man.netbsd.org/raidctl.8, realizing that Disklabel supports no
> larger than 2TB disks.
> So found this: https://wiki.netbsd.org/users/mlelstv/using-large-disks/
> and tried to follow the Raidframe section.
> Then i got the correct size i expected...
>
> The speed though, is slow! (again compared to zfs zraid)
> When i had FreeBSD installed, i did the following. Installed the system
> to the SSD.
> Then, did this, where **** is repeaded 4 times for the disks...
> # gpart destroy -F ada0 ****
> # gpart create -s gpt ada0 ****
> # gpart add -t freebsd-zfs ada0 ****
> # zpool create -f island raidz /dev/ada0p1 /dev/ada1p1 /dev/ada2p1
> /dev/ada3p1 (this was instant)
> island/scumm 21T 140K 21T 0% /zstorage/scumm
>
> This is the speed...
> # dd if=/dev/zero bs=1024k count=1000 of=/zstorage/scumm/test.txt
> 1048576000 bytes transferred in 0.259733 secs (4037123661 bytes/sec)
> # dd if=/dev/zero bs=4096k count=1000 of=/zstorage/scumm/test.txt
> 4194304000 bytes transferred in 1.534966 secs (2732505282 bytes/sec)
> # dd if=/dev/zero bs=1024 count=1000 of=/zstorage/scumm/test.txt
> 1024000 bytes transferred in 0.004667 secs (219403356 bytes/sec)
>
> Then on NetBSD, installed on the SSD i did this:
>
> # gpt destroy wd0 ****
> # gpt create -Af wd0 ****
> # gpt add -t raid -l raid5@wd0 -b $(( 2048 )) -s 15628051053 wd0 ****
>
> # raidctl -C /etc/raid5.conf raid5
> # raidctl -I 13371337 raid5
> # raidctl -iv raid5 (This took around 72+ hours)
> # gpt create -Af raid5
> # gpt add -a 1024 -t ffs -l island raid5
> /dev/rraid5: Partition 1 added: 49f48d5a-b10e-11dc-b99b-0019d1879648 34
> 46884152860
> # dkctl raid5 addwedge island 34 46884152860 ffs
> # newfs -O2 -b 65536 -s -1g /dev/dk7 (trying to enable
> https://man.netbsd.org/wapbl.4)
> # tunefs -l 1g /dev/dk7
> # mount -o log /dev/dk7 /mnt
> # umount /mnt
> # mount /dev/dk7 /mnt/island/
> /dev/dk7 22T 8.0K 21T 0% /mnt/island
>
> from /etc/fstab
> /dev/dk7 /mnt/island ffs rw,noatime,log 1 1
>
> This one, was done before initializing the raid, on one disk
> nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/test.txt
> 1048576000 bytes transferred in 2.201 secs (476408905 bytes/sec)
>
> Writing to the raid:
> nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/island/test.txt
> 1048576000 bytes transferred in 68.984 secs (15200278 bytes/sec)
> nas1# dd if=/dev/zero bs=4096k count=1000 of=/mnt/island/test.txt
> 4194304000 bytes transferred in 478.951 secs (8757271 bytes/sec)
> nas1# dd if=/dev/zero bs=1024 count=1000 of=/mnt/island/test.txt
> 1024000 bytes transferred in 0.054 secs (18962962 bytes/sec)
>
> If i tried bs=512k and count=1000, i get around 1.6Billion bytes per sek
> write, woohoo...
> Then try the same with count=2000 and get 1.3 million bytes per sek write
> hmmm ....
>
> This is /etc/raid5.conf
> START array
> # numRow numCol numSpare
> 1 4 0
>
> START disks
> /dev/dk3
> /dev/dk4
> /dev/dk5
> /dev/dk6
>
> START layout
> # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_5
> 32 1 1 5
>
> START queue
> fifo 100
>
>
> Summary:
> It seems like this is 200 times slower than FreeBSD zraid, is that
> reasonable? or where did i go wrong?!
> Again, only curious. Was thinking about installing OpenBSD, they have a
> more similar raid system compared to netbsd(more or less exactly the
> same:))
>
> Could it be worth adding a Big disk section to the raidctl manpage:
> https://man.netbsd.org/raidctl.8
>
> Thanks in advance!
> Br Nicklas
>
>
