[EMAIL PROTECTED] wrote on Thu, 02 Mar 2006 18:28 -0600:
> [EMAIL PROTECTED]:~/pvfs2$ inst1.4/bin/pvfs2-cp -t /tmp/bigfile3 /mnt/pvfs2/
> [E 17:58:03.909494] Error: encourage_send_incoming_cts: mop_id 1962 in
> CTS message not found.
No clue. I too have seen something similar that looks like some
sort of a race, although with different errors in the CTS path. Do
what you can to get a good debugging trace (PVFS2_DEBUGMASK=network)
on client and server and we figure it out. I assume this happens on
the openib version too? The bit of time I spent looking at it, I
found the problem goes away when debugging is on, how fun. I'm not
sure what changed to cause this to happen, perhaps some optimization
higher up (like immediate return from sends?) is causing BMI_IB
functions to be called more quickly.
> On another note:
> Pete, Is there any particular reasoning for your selection of the
> qp_init_attributes in ib_new_connection() [setup.c:119] ?
> And would there be any way to figure out if we're using all of the
> 'resources' we've specified using those settings?
> (I've had to change several of them to get them to work with my HCA's)
I stole them from a mellanox test code. With the new Arbel memfree
NICs, the numbers may have to be smaller. I don't think we really
need 40 or even 20 s/g entries anyway. The best way to do things is
to ask the HCA what it can provide, then use those numbers to
configure the QPs. At the time, VAPI did not have such a call, but
openib does have an HCA inquiry function we should use.
When you use up all the NIC's resources, it will complain with an
error (synch or async), so likely that is not the immediate problem.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
