On Fri, 31 Aug 2018 17:13:22 -0300, Marcelo Ricardo Leitner wrote: > On Tue, Aug 28, 2018 at 08:43:51PM +0200, Jakub Kicinski wrote: > > Ugh, CC: netdev.. > > > > On Tue, 28 Aug 2018 20:05:39 +0200, Jakub Kicinski wrote: > > > Hi! > > > > > > I wonder if we can use phys_port_id in switchdev to group together > > > interfaces of a single PCI PF? Here is the problem: > > On Mellanox cards, this is already possible via phys_switch_id, as > each PF has its own phys_switch_id. So all VFs with a given > phys_switch_id belong to the PF with that same phys_switch_id.
You mean Connect-X4 and on, Connect-X3 also shares PF between ports. But you're right, in simpler designs there is usually a 1:1 relation between an external networking port, a PCIe PF and a eswitch. Which does not mean it's correct to conflate the ports. > I understand this is a vendor-specific design It's hardly a strange design :) If the card is a true switch associating VFs on the PCIe side with an Ethernet port makes no sense. > but if you have the same phys_switch_id across PFs, does it really > matter on which PF the VF was created on? It does, because one can have multiple PFs, potentially connected to different hosts. And legacy NDOs, for example, must target the right one. So for one thing it seems like the right thing to do to associate VFs with PFs, not external ports. The other thing is that if you look at your Mellanox card in switchdev mode, you'll notice that systemd/udev is not able to name the representor interfaces properly without custom rules, because they are not linked to the PCIe PF. They are not linked because VF reprs don't provide legacy SR-IOV NDOs, so we prevent libvirt from poking at them. phys_port_id should steer libvirt away from VF representors, so we can get proper naming. (And having legacy NDOs implemented everywhere is a no-go, because ip link output grows quadratically.)
