On Mon, 4 Feb 2008, Muhammad Atif wrote:
I am trying to port xensockets to openmpi. In principle, I have the
framework and everything, but there seems to be a small issue, I cannot
get libevent (or OPAL) to give callbacks for receive (or send) for
xensockets. I have tried to implement native code for xensockets with
libevent library, again the same issue. No call backs! . With normal
sockets, callbacks do come easily.
So question is, do the socket/file descriptors have to have some special
mechanism attached to them to support callbacks for libevent/opal? i.e
some structure/magic?. i.e. maybe the developers of xensockets did not
add that callback/interrupt thing at the time of creation. Xensockets is
open source, but my knowledge about these issues is limited. So I though
some pointer in right direction might be useful.
Yes and no :). As you discovered, the OPAL interface just repackages a
library called libevent to handle its socket multiplexing. Libevent can
use a number of different mechanisms to look for activity on sockets,
including select() and poll() calls. On Linux, it will generally use
poll(). poll() requires some kernel support to do its thing, so if
Xensockets doesn't implement the right magic to trigger poll() events,
then libevent won't work for Xensockets. There's really nothing you can
do from the Open MPI front to work around this issue -- it would have to
be fixed as part of Xensockets.
Second question is, what if we cannot have the callbacks. What is the
recommended way to implement the btl component for such a device? Do we
need to do this with event timers?
Have a look at any of the BTLs that isn't TCP -- none of them use libevent
callbacks for progress. Instead, they provide a progress function as part
of the BTL interface, which is called on a regular basis whenever progress
needs to be made.
Brian
