[EMAIL PROTECTED] wrote on Mon, 10 Mar 2008 13:51 -0500:
> I am trying to hack together a test case to implement what we had
> talked about in the previous emails with a wr_credit...
> I'm trying to keep track of it in the openib_device (od) structure
> inside openib.c and would like to keep the necessary changes inside
> openib.c if at all possible. The problem I'm running into, is that
> I'm going to need to call check_cq() from inside the send_rdma writes
> function, which lies in openib.c, not ib.c. openib.c has a
> function for this but its really intended to work *with* ib.c's
> check_cq() fucntionality...
> In order to get around this I needed to make ib_check_cq() visible to
> openib.c (got rid of the static and added a declaration to ib.h)..
> but I'm getting weird things when I'm linking..
>
> Any ideas how to get around this?
>
> lib/libpvfs2-server.a(bmi-server.o):(.rodata+0x780): undefined
> reference to `bmi_ib_ops'
> collect2: ld returned 1 exit status
> make: *** [src/server/pvfs2-server] Error 1
>
> (I've attached a very rudimentary patch that sort of gets at what I'm
> trying to do, not sure if its correct yet, still trying to compile)
Just hack up anything you like to get it to work. If it fixes the
situation, we'll go back and clean up the code later.
It is optimistic, what you're trying to do, but I'm not sure if it
will be sufficient. If there are no credits to get back from
checking the CQ, you'll just deadlock. I'm also nervous about
locking implications, as you're checking the CQ in the thread that
is trying to do the send. Not sure if we have done this before.
A simpler way would be just to just fail whatever operation got us
into this RDMA, by abandoning it, with another state that says we're
waiting on credits. An easier first step is just to add lots of
printfs to track the credits and see if you can correlate a credit
overflow with the rdma failures. If that works, a check at the top
of "post rdma" can say whether we should even bother and we won't
need your fixup step of looking at the CQ from the send.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
