On Tue, 2008-01-01 at 15:28 -0800, Old Apple wrote: > With OFED 1.2.5.4, everything connects without any > problem, all 6 disks are seen inside the server. By > setting max_sect to 16384 and srp_sg_tablesize to 256 > we however obtain only 110 MB/sec per LUN which is too > low for our DDN system. (To make sure we are not > simply piping everything into one SDR port of DDN, we > tried to use only 3 LUNS together, and still never > went over 110 MB/sec per LUN).
As a quick hack/workaround, add max_cmds_per_lun=5 to the string you are echoing to add-target under OFED. That should get you better performance, as the SRP implementation will send requests to the array when it has no credits available to do so. For a better fix that will let you queue commands to the DDN for a single lun when the others are not busy, I sent some patches a weeks ago to respect the credit handling, and to force a maximum queue length. Search for subjects: IB/srp: respect target credit limit IB/srp: use scatter gather chaining IB/srp: allow user to control host queue length The one for scatter gather chaining may not work for pre 2.6.24-rc1 kernels, I haven't checked. It isn't absolutely critical. With the third one, "...queue length", you can replace the max_cmds_per_lun=5 with "queue_len=31" to prevent some performance degradation at full tilt -- I'm still tracking it down, but it wasn't as bad as seeing 110MB/s, IIRC. You may want to keep the max_cmds_per_lun, though, if your workload is such that all 6 LUNs will be equally busy. With these, I can max out a port on the DDN with a single LUN (~750MB/s). _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
