2010/6/12 Dotan Barak <[email protected]>: > On 12/06/2010 03:22, Ding Dinghua wrote: >> >> 2010/6/11 Dotan Barak<[email protected]>: >> >>> >>> Hi. >>> >>> On 11/06/2010 10:51, Ding Dinghua wrote: >>> >>>> >>>> Hi all: >>>> I'm using RDMA to do fs-metadata mirror between nodes. I >>>> encountered a strange problem when the program was running: >>>> Complete queue handler reported that the RDMA-Write operation failed, >>>> the status of corresponding "struct ib_wc" is "IB_WC_RETRY_EXC_ERR". >>>> The problem is encountered randomly. I don't know the meaning of this >>>> error code as well as what to do next. Would anyone give me some tips? >>>> thanks a lot. >>>> >>>> >>> >>> Do you sync between the sides before closing the QPs? >>> >> >> Can you say it more detail? thanks. >> > > If you try to send a message from local QP to a remote QP before the remote > QP is in RTR state (or after it was closed/transferred to the ERROR state), > you may get RETRY EXCEEDED, because there isn't any QP in the remote side > that can accept your message (and send a response). > > How do you connect the QPs? (And how do you close the connection between > them) > Sorry i forget the close issue.
1. Local node call ib_poll_cq to process the remaining complete queue entry, 2. Local node call rdma_disconnect to destroy connection, before remote side ack, the thread will wait on a wait queue. 3. After catching this request, the remote node will also call ib_poll_cq to process the remainning complete queue entry, then do some resource-release work, then send a reply. 4. Local node was waken up and do resource-release work. > Dotan > -- Ding Dinghua -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
