I've the impression that 5.1's fs write performance on RAID5s is exceptionally poor.
On a 5.1 system with a RAID5 accross three 10k discs, un-taring src.tgz takes 161s with WAPL and 347s with softdep. On the same system, but with a RAID1 accross the same three discs, it takes only 41s. On that system, dd'in /dev/zero to a file gives 15MB/s on the RAID5 and 41MB/s on the RAID1 (both on the same three 10k discs). On 4.0.1 with softdeps, it takes 22s on a RAID1 on two 15k discs and 29s on a RAID5 accross eight 10k discs. On these 4.0.1 systems, the same dd gives 20MB/s on the RAID5 and 78MB/s on the RAID1. So while the device throughput ratio for my 5.1 vs 4.0.1 RAID5s is 3:4, the fs througput is more like 1:5 (log/softdep) or 1:11 (softdep/softdep). Also, on the 5.1 system on identical discs, file system throughput on RAID5 vs. RAID1 is 1:4, while, for the RAID1, 50% more data need to be written (it's three discs). Is there something wrong with my 5.1 system (amd64, Adaptec 19160, three Seagate 10k/36G Cheetah drives) or can someone confirm this behaviour?
