On Wed, Dec 12, 2007 at 04:08:31PM +0200, Pavel Shamis (Pasha) wrote:
> Gleb Natapov wrote:
>> On Wed, Dec 12, 2007 at 03:37:26PM +0200, Pavel Shamis (Pasha) wrote:
>>
>>> Gleb Natapov wrote:
>>>
>>>> On Tue, Dec 11, 2007 at 08:16:07PM -0500, Jeff Squyres wrote:
>>>>
>>>>> Isn't there a better way somehow? Perhaps we should have "select"
>>>>> call *all* the functions and accept back a priority. The one with the
>>>>> highest priority then wins. This is quite similar to much of the
>>>>> other selection logic in OMPI.
>>>>>
>>>>> Sidenote: Keep in mind that there are some changes coming to select
>>>>> CPCs on a per-endpoint basis (I can't look up the trac ticket right
>>>>> now...). This makes things a little complicated -- do we need
>>>>> btl_openib_cpc_include and btl_openib_cpc_exclude MCA params to
>>>>> include/exclude CPCs (because you might need more than one CPC in a
>>>>> single job)? That wouldn't be hard to do.
>>>>>
>>>>> But then what to do about if someone sets to use some XRC QPs and
>>>>> selects to use OOB or RDMA CM? How do we catch this and print an
>>>>> error? It doesn't seem right to put the "if num_xrc_qps>0" check in
>>>>> every CPC. What happens if you try to make an XRC QP when not using
>>>>> xoob? Where is the error detected and what kind of error message do
>>>>> we print?
>>>>>
>>>>>
>>>> In my opinion "X" notation for QP specification should be removed. I
>>>> didn't want this to prevent XRC merging so I haven't raced this point.
>>>> It is enough to have two types of QPs "P" - SW credit management "S" -
>>>> HW credit management.
>>> How will you decide witch QP type to use ? (SRQ or XRC)
>>>
>>>
>> If both sides support XOOB and priority of XOOB is higher then all other
>> CPC
>> then create XRC, otherwise use regular RC.
>>
> If some body have connectX hca but he want to use SRQ and not XRC ?
This will be the default. (prio of OOB will be bigger than of XOOB), but
if uses will want to use XRC it will increase XOOB priority by
specifying MCA parameter.
> I guess anyway we will be need some additional parameter that will allow
> enable/disable XRC, correct ? (So why just not leave the X qp type ?)
Because we want to support mixed setups and create XRC between nodes that
support it and RC between all other nodes.
--
Gleb.
