> But whacky situations might occur in a multithreaded application where > one thread calls free() while another thread calls malloc(), gets the > same virtual address that was just free()d but has not yet been > unregistered in the kernel, so a subsequent ibv_post_send() may > succeed but be sending the wrong data. > > Put simply: in a multi-threaded application, there's always the chance > that the notify won't get to the user-level process until after the > global notifier variable has been checked, right? Or, putting it the > other way: is there any kind of notify system that could be used that > *can't* create a potential race condition in a multi-threaded user > application?
Without thinking too much about the proposal (except that it adds a lot of new verb interfaces and a lot of kernel code, and therefore feels like a hassle to me), I don't see how this race is solved by moving a cache to the kernel. If you have free()/malloc() of a buffer running in parallel with send operations targeting the same buffer, then that seems like a buggy MPI application. Since free()/malloc() might not involve the kernel at all (the userspace library might keep its own free list, etc) I don't see how a registration cache in the kernel would help anyway. Now, since free()/malloc() operations must be serialized with respect to send/receive operations in userspace anyway, I don't see why a simpler (and possibly more flexible/powerful) kernel notifier design can't work -- if free() releases virtual memory back to the kernel, then the kernel notifier will run before the free() call returns, so things should work as planned. - R. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
