Hi Moni/Shachar,
Thank you for your feedback/comments. I have tried to address some of
your concerns /comments inline below.
> Few comments to the IB/core part
>
> 1. RoCEv2 spec (Annex A17) sees RoCE type (or as the patch name
> it - network_hdr_type) as an attribute of the source GID. If so, there is no
> point to for address structures (ib_ah_attr, ib_sa_path_rec, …) to have both
> sgid and RoCE type. The alternative is to keep an extra info in each GID table
> entry about the RoCE header (IPv4/IPv6 or GRH) and use it when modifying
> QP from INIT to RTR (RC/UC) or creating AH (UD).
Yes, that is correct.
The overarching goal behind this patch is to keep RDMA-CM as the central entity
that decides the protocol (ROCEV2 /ROCEV1)
and hides all the address resolution details from applications.
This is just a continuation of the existing RDMA-CM design philosophy for IP
Based GIDs that lets applications operate over any form
of RDMA Service in a completely transparent way without any changes to the
applications themselves.
Protocol Resolution(RoCEV2 or V1) must be done even before we
send out the 1st connection request packet, the corresponding MAD pkt must be
V2 for a destination that is behind a router, correct?
I am not sure we want to try sending out RoCEv2 connection requests first and
if that fails ,then fallback/retry with RoCEv1 request.
That would needlessly complicate and slow down the connection-establishment
process, do you agree ?
Instead it's much more seamless to take help of the IP stack to select the
network header type and subsequently the ROCE pkt format while
attempting to establish the connection.
The one thing that my patch does miss out though is the ability for local
subnet end nodes to be able to communicate using RoCEv2 packet formats.
(The default policy in this patch is to use ROCEV1 for nodes connected directly
/within the same local subnet.)
I believe that can be achieved by extending RDMA-CM to override the default
protocol selection policy proposed in this patch by making use of
the GID Entry attribute type in another patch.
This scheme should still work with your proposal in point# 2 , where GID Table
management would move to a central/stack module instead of being
vendor specific.
Thanks
Som
> 2. Population of table in for RoCE should be a common code to
> all drivers and needs to be moved to IB/core (unlike today when it is specific
> to a vendor). We plan to push such a change soon.
>
> 3. If I get it right, the purpose of the change in ib_addr is to
> that determine RoCE type based on the resolution of address from OS.
> If gateway is used (dest address is not in the subnet of source
> address) then the protocol will be V2, otherwise it will be V1. This means
> that
> IB over UDP is not possible on the same network. I don't think that this is
> acceptable.
