Sorry for the massive lag in this conversation - between trying to balance working with Linux-RDMA with the rest of my job, support problems and vacations, this got pushed to the bottom of my queue while I thought about the best approach to the issue.
When we left this, we were discussing the most appropriate API, with Jason suggesting we go through /dev/*umad* and Sean suggesting leveraging the existing async functions in libibverbs: > Re: Proposal for adding ib_usa to the Linux Infiniband Subsystem > Jason Gunthorpe > Fri, 21 May 2010 12:46:22 -0700 > > My thinking was sort of: > > fd = open("/dev/..umad.."); > ioctl(fd, SUBSCRIBE_TRAP, &trap_description) > read(fd..); // Events arrive > close(fd); // Event is unsubscribed > > The reason this fits nicely with umad is that you really actually want > the GMP trap, not some processed version.. So this is like a multicast > subscribe in IP. and... > RE: API for Proposal for adding ib_usa to the Linux Infiniband Subsystem > Hefty, Sean > Tue, 08 Jun 2010 14:23:59 -0700 > > Would something like this work for you? > enum ibv_event_type { > ... > + IBV_EVENT_SA > ... > > struct ibv_async_event { > union { > + struct ibv_sa_event *sa_event; > ... > + int ibv_reg_sa_event(struct ibv_context *context, uint8_t port_num, > uint16_t trap_number); > > This would tie in with the libibverbs async event reporting and provide a > simple registration API. (Deregistering would just be done automatically > when > closing the ibv_context.) I do like Sean's idea quite a lot since it's simple and ties into an existing async interface rather than requiring a new one - but I've also been wondering about security issues and whether it makes sense for most SA/SM traps to be exposed to end-user applications. Here's what I mean: the list of traps includes GID in/out of service, MC group traps, Path is no longer valid, security notifications like pkey errors, etcetera. However, only 6 of these are valid for the SA/SM even in theory and it's not clear to me that anything except GID in/out of service and MC group created/deleted are actually implemented in the OpenSM. Plus, I don't think unprivileged applications should have access to the other traps in any case. So, what I'm wondering is should we simplify things so that the application can only register for GID in/out notifications? If we did this correctly, it would be easy to extend the interface later to add other trap types if the need becomes clear. So, my suggestion for the API would be more like this: enum ibv_event_type { ... + IBV_EVENT_GID ... struct ibv_async_event { union { + struct ibv_gid_event *gid_event; ... + int ibv_reg_gid_event(struct ibv_context *context, uint8_t port_num); -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html