After a bit of experimentation, it turns out that diag 2a8 is actually effectively half duplex (while you can effectively send and receive in the same operation, the call doesn't return until all the operations complete), so we detected that properly. We also set the interface driver to report 100Gbit as maximum theoretical throughput for server to VSWITCH connections to handle the in-memory only case. The VSWITCH seems to cope, and the scheduling stuff in Solaris seems to like it, so we'll see what happens until someone complains.
BTW, Alan, I got the wrong acronym - brain failure in my aging. ECN is the correct one (explicit congestion notification). Allows a layer 2 device to explicitly notify senders that the target is congested and to back off on sending rates to try to ease congestion or switch to another path if one is available. Would be very handy for this use....
