On Wed, Oct 28, 2009 at 10:58:58AM -0700, Sean Hefty wrote: > >I guess I was hoping you'd be able to do the private data in userspace > >and avoid those kind of special cases.. > > The private data would still be done in user space. This allows > apps that make use of ADDR_CHANGE events to continue working as > normal, while still allowing for address resolution to be done in > user space. This should meet Or's request. User space may not know > if they can resolve a particular address before binding.
Well, looking at this, I see that ADDR_CHANGE is only generated on netdev events NETDEV_BONDING_FAILOVER - so what are the semantics WRT to UCM here? As I said before, defeating the ND process seriously affects how bonding works. I think if UCM is going to have bonding-like HA it has to do the work itself. It has to record the bonding IP is bound to multiple GID/LIDs and it has to attempt connecting to multiple addrs. So, I don't see ADDR_CHANGE fitting into this. An IB centric IB connection lost event to trigger reconnect would be more appropriate. This would be desirable anyhow for AF_IB only apps, so multi-protocol apps can handle both cases - the same recovery action is probably required anyhow... FWIW, I think HA + UCM is very doable, if UCM can construct multiple PRs based on bonding data it collects it should pass them into the kernel using the mechanism we talked about for APM. The kernel does not have to implement APM but it certainly can run through the combinations and use the first to reply to the GMP as the path to establish a connection. For typical 2 port bonding there would be 4 combinations, 2 of which the kernel could eliminate immediately by detecting the local port is downed. Then again, is bonding + ucm going to be a common usage model? Without the faster address failover provided by the unsolicted ND packet the recovery time will be worse. On the other hand, if all the above is done UCM would actually have enough information to setup multi-port APM... Anyhow, I don't suggest you try to implement HA + UCM out of the gate, but as a direction for work in that area to take I think AF_IB only plus IB_LINK_LOST event, plus 'APM-lite' kernel PR round robin is a viable approach. Keeping AF_IB uniformly as AF_IB seems like it would make things simpler to implement and describe WTF it is doing. > At this point, I'm guessing that the only thing that would prevent > this from working is if an explicit check is added. Some kinds of checks are needed, passing AF_INET and AF_INET6 into rdma_resolve_ip will mis-function. dst=AF_IB will have to branch off before this. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
