On Thu, 19 Sep 2013 20:14:33 +0200 Edgar Fuß <e...@math.uni-bonn.de> wrote:
> > options RAIDOUTSTANDING=40 #try and enhance raid performance. > Is there any downside to this other than memory usage? > How much does one unit cost? This is from the comment in src/sys/dev/raidframe/rf_netbsdkintf.c : /* * Allow RAIDOUTSTANDING number of simultaneous IO's to this RAID device. * Be aware that large numbers can allow the driver to consume a lot of * kernel memory, especially on writes, and in degraded mode reads. * * For example: with a stripe width of 64 blocks (32k) and 5 disks, * a single 64K write will typically require 64K for the old data, * 64K for the old parity, and 64K for the new parity, for a total * of 192K (if the parity buffer is not re-used immediately). * Even it if is used immediately, that's still 128K, which when multiplied * by say 10 requests, is 1280K, *on top* of the 640K of incoming data. * * Now in degraded mode, for example, a 64K read on the above setup may * require data reconstruction, which will require *all* of the 4 remaining * disks to participate -- 4 * 32K/disk == 128K again. */ The amount of memory used is actually more than this, but the buffers are the biggest consumer, and the easiest way to get a ball-park estimate.... So if you have a RAID 5 set with 12 disks and 32K/disk for the stripe width, then for a *single* degraded write of 32K you'd need: 11*32K (for reads) + 32K (for the write) + 32K (for the parity write) which is 416K. If you want to allow 40 simultaneous requests, then you need ~16MB of kernel memory. Maybe not a big deal on a 32GB machine, but 40 probably isn't a good default for a machine with 128MB RAM. Also: remember that this is just one RAID set, and each additional RAID set on the same machine could use that much memory too.... Also2: this really only matters the most in degraded operation. I can't think, off-hand, of any downsides other than memory usage.. Later... Greg Oster
