Hmm, I do amortize the cost of ibv_ack_cq_event() over multiple
ibv_get_cq_event() calls; however when the shutdown is in progress I
don't have the last event that was "gotten" so I have to call
ibv_get_cq_event() one last time to get an event to acknowledge against.
I suppose it's probably better to keep the last event processed if it
hasn't been acknowledged and use it to issue the final acknowledge when
shutting down. Then I wouldn't have to make that ibv_get_cq_event() call.
One last question, when I create the completing event queue I set it to
non-blocking but I find that during shutdown I have to do that again
before making the final call to ibv_get_cq_event() otherwise it blocks.
Which I suppose is why it returns EAGAIN when there are no pending
events, but I don't understand why I have to set it to non-blocking again.
Anyway, much thanks for all your help.
Nitin
Sean Hefty wrote:
I guess my question is, what's the best way to destroy IB resources? (Perhaps
even, what's the best way to init them in the first place).
If you're destroying the CQ, there's no need to call ibv_get_cq_event() or
ibv_poll_cq(), unless you need completion information (for example, from flushed
receives).
However, every successful call to ibv_get_cq_event() needs a corresponding call
to ibv_ack_cq_event(). You can call ack(1) for each cq event, or count the
number of times that get returns success and call ack(get_cnt) once before
calling destroy. Note that the count refers to the number of cq events, and not
the number of completions returned through ibv_poll_cq.
For your drain_cq() function, you should be safe doing something like this:
while (ibv_poll_cq(...) > 0)
/* optional processing of any left over completions */;
ibv_ack_cq_event(...this_cqs_total_event_cnt); /* or ack after get */
ibv_destroy_cq(...);
ibv_dealloc_pd(), ibv_destroy_cq() and ibv_destroy_comp_channel() all return
error EBUSY
This sounds like a QP isn't being destroyed. I'm not sure that anything else
fails CQ destruction with EBUSY.
Btw, if you're using the rdma_cm interface, then it's simpler to use the
rdma_create_qp/rdma_destroy_qp calls, which allows the rdma_cm to perform the QP
state transitions for you.
- Sean
------------------------------------------------------------------------
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 8.5.392 / Virus Database: 270.13.56/2302 - Release Date: 08/14/09 06:10:00
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
