On Thu, Oct 15, 2015 at 7:58 PM, Hefty, Sean <[email protected]> wrote: >> >> ib_create_ah_from_wc needs to resolve the DMAC in order to create the >> >> AH (this may result sending an ARP and waiting for response). >> >> CM uses this function (which is now sleepable). >> > >> > This is a significant change to the CM. The CM calls are invoked >> assuming that they return relatively quickly. They're invoked from >> callbacks and internally. Having the calls now wait for an ARP response >> requires that this be re-architected, so the calling thread doesn't go out >> to lunch for several seconds. >> >> Agree - this is a significant change, but it was done a long time ago >> (at v4.3 if I recall). When we need to send a message we need to > > We're at 4.3-rc5? >
Sorry, meant v3.14. >> figure out the destination MAC. Even the passive side needs to do that >> as some vendors don't report the source MAC of the packet in their wc. >> Even if they did, since IP based addressing is rout-able by its >> nature, it should follow the networking stack rules. Some crazy >> configurations could force sending responses to packets that came from >> router1 to router2 - so we have no choice than resolving the DMAC at >> every side. > > Ib_create_ah_from_wc is broken. It is now an asynchronous operation, only > the call itself was left as synchronous. We can't block kernel threads for a > minute, or however long ARP takes to resolve. The call itself must change to > be async, and all users of it updated to allocate some request, queue it, and > handle all race conditions that result -- such as state changes or > destruction of the work that caused the request to be initiated. Today, cm assumes paths are reversible primary_path->reversible = 1. That's true for both IB and RoCE. We could say vendors must report the SMAC in WC and then ib_create_ah_from_wc will be atomic (for these cases). If we wish to lift these limitations, we need to make ib_create_ah_from_wc asynchronous, but that's true even prior the RoCE IP based addressing patch. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
