> ----- Message from "Or Gerlitz" <[EMAIL PROTECTED]> on Thu, 01 Feb 2007 11:17:53 +0200 ----- > > Dotan Barak wrote: > > I think that now, when implementation of IPoIB CM is available and SRQ
> > is being used, one may > > need to use a SRQ with more than 16K WRs. > > IPoIB UD uses SRQ by nature (since RX from all peers consume buffers > from the --only-- RQ) and lives fine with 32 buffers (or 64 you can look > in the code). Moreover, my assumption is that > > pps(RC) <= pps(UC) <= pps(UD) > > this means that what ever number of RX buffer for UD/2K MTU which is > "enough" to have no (or close to zero) packet loss under some traffic > pattern, the same pattern can be served with IPoIB CM using SRQ of the > same size. I would expect that you will need more than 32 or 64 buffers using RC and SRQ. With larger packets it takes longer to do receive processing on each packet under RC. Larger packets means it takes more time to do checksum and copy to the socket because of up to 60K or data vs. 2K. The residency time on the receive queue will be longer. In the traffic pattern where one adapter is receiving from many adapters over the fabric, there will be a larger imbalance between sender rate vs. the receiving rate out of the queue. Given a large enough TCP send and receive window for a single socket to get peak bandwidth, muliple sockets will have more packet in flight for a single destination at the same time in this pattern > > Or. > > > Bernie King-Smith IBM Corporation Server Group Cluster System Performance [EMAIL PROTECTED] (845)433-8483 Tie. 293-8483 or wombat2 on NOTES "We are not responsible for the world we are born into, only for the world we leave when we die. So we have to accept what has gone before us and work to change the only thing we can, -- The Future." William Shatner
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
