Here are some more details on the ndrping issues that I mentioned during the con-call this morning. (Btw, the test fails with IBAL as well.)
The test fails if the client runs on mthca, and the server runs on mlx4. The client code calls CreateEndpoint() using values of 0 for inbound and outbound RDMA reads. The 0's get mapped into the CM REQ messages. The server calls CreateEndpoint() using the local HW max values. These end up getting pushed into the CM REP message. Both the client and server QPs get configured using the value carried in the REP. I'm not sure where the best place is for a fix. ndrping must at least be changed to pass in non-zero values (I think in place of InboundReadLimit) on the client side. It should probably be changed on the server side to call GetConnectionData() to check the provided values and select the minimum. Also, there seems to be a bug in the mthca driver that allows the QP to be configured with values that are too large, which results in the QP being unable to send data. Note that I'm running the test using RDMA writes, not reads, and the first send from the client is what never completes. ndrping works if I hard code the limits to 1 on both sides of the connection. - Sean _______________________________________________ ofw mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
