> The overall problem is that the throttle is set based on the > specifications, some would say limitations, of the storage array. We can > throttle at the LUN level if we know the characteristics of the LUN. The > sd code does an internal table lookup and then looks at sd.conf to set > paramaters if any are present. > > http://cvs.opensolaris.org/source/xref/on/usr/src/uts/common/io/scsi/targets/sd.c#550 > is the start of the static table. > > The fun part is that a lot of arrays have a per array queue, or a per > port queue, or a per LUN queue and in the worst cases a combination of > the above. In that case you have to do the math and use the smallest > value for the big stick - sd_max_throttle.
Hmm. Interesting. I notice that the "SEAGATE_THROTTLE" is defined to 15 for a set of known disks (ST19171FC, ST136403FC) that I've been having some troubles (more with the ST136403FC) using in Sun A5000 arrays at various times - when the load gets high towards those disk they start resetting and going on/off from the bus causing all kinds of problems... I even tried manually setting the sd_max_throttle to insanely low levels without it helping. This is not LUN-related though since these disks are directly attached in the A5000 boxes. The only way I've found that makes the errors go away was to find FC HBA drivers's that would allow me to limit the number of tagged commands that was sent to the disk, or drivers that had low enough defaults... For example with the Qlogic QLA2100 default qla2100.conf file settings of: # Maximum number of commands queued for each port in the adapter. hba0-max-iocb-allocation=256; # Maximum number of commands queued on each logical unit. hba0-execution-throttle=31; Things would go downhill as soon as the server went to multiuser and started serving NFS traffic. By limiting both parameters to 7 the errors disappeared and things work smoothly (albiet possibly slower than it could be running - not really tested). I haven't really tested what I can increase the numbers to before things start to break yet (this is a $HOME fileserver that's in use after all). (We got similar problems when these arrays used to be attached to an Ultra2 with SBus FC HBA cards - we couldn't get it to run stable until we found some JNI 3:d party HBA cards - but never with the 36GB disks... However I just might give the 36GB:ers another shot now when we use the QLA2100 PCI cards instead and can adjust these parameters...) I It almost _feels_ like the sd_throttle_max value isn't really used for these FC disks since my problem only goes away then I limit things in the FC HBA driver and *not* in the SD driver. (Both the QLA2100 and the JNI drivers use the SD driver and not the SSD driver btw) This message posted from opensolaris.org _______________________________________________ storage-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/storage-discuss
