On Thu, Oct 25, 2007 at 10:55:25AM -0400, Jeff Squyres wrote:
> On Oct 25, 2007, at 10:35 AM, Gleb Natapov wrote:
>
> > I don't think xrc should be used by default even if HW supports it.
> > Only if
> > special config option is set xrc should be attempted.
>
> Why?
XRC is a crippled RC protocol for scalability sake. Its use makes
progress of one process depend on behaviour of other processes on the
same node which make cause different interesting effects. And of cause
SW flow control is not possible when using XRC, so for small jobs it
will be actually slower. I don't thinks it is wise to use XRC over
regular RC if there is a choice.
>
> > And xrc availability
> > can be tested in runtime without additional options in ini file.
>
> Is there a flag on the device / port that indicates XRC availability?
XRC requires creation of special kind of QP. If this fails XRC is not
available.
>
> > I don't know iWarp enough to tell if it is possible to find out in
> > runtime if rdma_cm is mandatory or other means of connection
> > establishment can be used, but if there is no way to do it, then new
> > parameter "hca_type" could be added to ini file with two possible
> > values "ib" and "iwarp".
>
> Yes, there is a flag on either the device or port (I forget which)
> which indicates whether it's an iwarp or IB device. I think (at
> least for today) we can assume that all iWARP devices require RDMA CM
> -- right, iWARP guys?
Great! Then I don't see the need to add parameters to ini file.
>
> So do you want the arbitration rules for which CPC to be used to be
> hard-coded in the openib component (possibly overridden by MCA
> parameter to force a specific selection)?
>
Not hard-coded, but controlled by regular mca mechanism, with default
behaviour dependant on HCA type. Not something new. We have this with
SRQ.
--
Gleb.
