Hi Stephen,
> The rdma-core stuff is likely to be a problem on Azure. The Mellanox
> device is always hidden as a secondary device behind a synthetic virtual
> device based on VMBus. There are two DPDK different ways this is used. One
> is with vdev_netvsc/failsafe/tap and the other is with netvsc PMD.
> In either case, the virtual device expects to see the MLX device show up
> as a VF if enabled.
> So unless you want to write native VPP drivers for VMBus as well, I don't
> see how having native rdma-core driver will help. It will only get
> confused.
I'd be interested to better understand how it works. My current understanding:
- the Mellanox device shows up as a PCI SR-IOV VF in the Linux VM ("the VF")
- the VF is associated with a VMBUS (not PCI) netvsc virtual device
- the netvsc and the VF shares the same MAC. The netvsc is guaranteed to be
always there whereas the VF can disappear between migrations etc.
- control plane traffic (multicast eg. ARP, other?) always go through the
netvsc
- if the VF is present is gets all the traffic minus control plane. If it is
not present, all traffic gets rerouted through netvsc as a fallback
Am I correct? Could you elaborate a little bit more about what is the control
plane traffic (basically what we should miss on the VF)?
If so, I see at least 2 problems we need to tackle:
- we need to receive control plane traffic from netvsc because of eg. ARP.
However, being control-plane/fallback, it could be slow and use eg. TAP or even
AF_PACKET. We will also need to use it to populate the neighbor entries of the
VF
- the VF can appear/disappear w/o notice in case of migration etc. That could
be handled in different ways, eg. by the control plane agent (removing the rdma
iface from VPP prior to migration, adding it after post migration) or by
leveraging bonding or...
Thanks for your help,
Ben
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12761): https://lists.fd.io/g/vpp-dev/message/12761
Mute This Topic: https://lists.fd.io/mt/30795618/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
