Jeff Squyres wrote:
On Jul 15, 2008, at 7:30 AM, Ralph Castain wrote:
Minor clarification: we did not test RDMACM on RoadRunner.
Just for further clarification - I did, and it wasn't a particularly
good
experience. Encountered several problems, none of them overwhelming,
hence
my comments.
Ah -- I didn't know this. What went wrong? We need to fix it if
there are problems.
RDMACM, on the other hand, is *necessary* for iWARP connections. We
know it won't scale well because of ARP issues, to which the iWARP
vendors are publishing their own solutions (pre-populating ARP caches,
etc.). Even when built and installed, RDMACM will not be used by
default for IB hardware (you have to specifically ask for it). Since
it's necessary for iWARP, I think we need to build and install it by
default. Most importantly: production IB users won't be disturbed.
If it is necessary for iWARP, then fine - so long as it is only used if
specifically requested.
However, I would also ask that we be able to -not- build it upon
request so
we can be certain a user doesn't attempt to use it by mistake ("gee,
that
looks interesting - let Mikey try it!"). Ditto for ibcm support.
Pasha added configure switches for this about a week ago:
--en|disable-openib-ibcm
--en|disable-openib-rdmacm
I like these flags but I thought there was going to be a run time check
for cases where Open MPI is built on a system that has ibcm support but
is later run on a system without ibcm support.
-DON
