On Thu, Sep 21, 2017 at 11:17:59AM +0200, Adrien Mazarguil wrote: > On Wed, Sep 20, 2017 at 06:33:43PM +0100, Kevin Traynor wrote: > > On 09/08/2017 10:56 AM, Loftus, Ciara wrote: > > >> Hi, > > >> > > >> I have compiled and built ovs-dpdk using DPDK v17.08 and OVS v2.8.0. The > > >> NIC that I am using is Mellanox ConnectX-3 Pro, which is a dual port 10G > > >> NIC. The problem with this NIC is that it provides only one PCI address > > >> for > > >> both the 10G ports. > > >> > > >> So when I am trying to add the two DPDK ports to my br0 bridge > > >> > > >> # ovs-vsctl --no-wait add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk > > >> options:dpdk-devargs=0002:01:00.0 > > >> > > >> # ovs-vsctl --no-wait add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk > > >> options:dpdk-devargs=0002:01:00.0 > > >> > > >> The port dpdk1 is added successfully and able to transfer data, but > > >> adding > > >> dpdk0 to br0 fails: > > >> > > >> 2017-09-06T14:19:20Z|00045|netdev_dpdk|INFO|Port 0: e4:1d:2d:4f:78:60 > > >> 2017-09-06T14:19:20Z|00046|bridge|INFO|bridge br0: added interface dpdk1 > > >> on > > >> port 1 > > >> 2017-09-06T14:19:20Z|00047|bridge|INFO|bridge br0: added interface br0 > > >> on > > >> port 65534 > > >> 2017-09-06T14:19:20Z|00048|dpif_netlink|WARN|Generic Netlink family > > >> 'ovs_datapath' does not exist. The Open vSwitch kernel module is probably > > >> not loaded. > > >> 2017-09-06T14:19:20Z|00049|netdev_dpdk|WARN|'dpdk0' is trying to use > > >> device > > >> '0002:01:00.0' which is already in use by 'dpdk1' > > >> 2017-09-06T14:19:20Z|00050|netdev|WARN|dpdk0: could not set > > >> configuration > > >> (Address already in use) > > >> 2017-09-06T14:19:20Z|00051|bridge|INFO|bridge br0: using datapath ID > > >> 0000e41d2d4f7860 > > >> > > >> > > >> With OVS v2.6.1 I never had this problem as dpdk-devargs was not > > >> mandatory > > >> and just specifying port name was enough to add that port to bridge. > > >> > > >> Is there a way to add port both ports to bridge ? > > > > > > It seems the DPDK function rte_eth_dev_get_port_by_name() will always > > > return the port ID of the first port on your NIC, when you specify the > > > single PCI address and that's where the problem is. There doesn't seem to > > > be a way currently to indicate to the calling application that in fact > > > two (or more) port IDs are associated with the one PCI address. > > > > > > I am cc-ing DPDK users mailing list for hopefully some input. Are there > > > any plans for the rte_eth_dev_get_port_by_name function to be compatible > > > with NICs with multiple ports under the same PCI address? > > > > > > > Hi Adrien/Nelio, > > > > Is this something you can answer? We're wondering how to handle this in > > OVS and whether a temporary or long term solution is needed. > > > > The original thread started here: > > https://mail.openvswitch.org/pipermail/ovs-dev/2017-September/338418.html > > Hi, > > I'm late to this thread and not too familiar with openvswitch's internals, > however after reading this thread here are some thoughts. > > About rte_eth_dev_get_port_by_name(), contrary to other PMDs, mlx4 generates > for each port something that is not a PCI bus address precisely to avoid > collisions. The name is the concatenation of the associated Verbs device > name and the physical port number, e.g. "mlx4_0 port 0" instead of > "0000:02:42.0". > > Referring to a port with this name through the above function should yield > the correct device.
Well, I replied before checking the current code. Interpret the above "should" as "would have been nice if it". The device goes by several names, and since commit [1] the name set by the PMD through rte_eth_dev_allocate() (struct rte_eth_dev.data->name) is not the same as the one used by rte_eth_dev_allocated() and rte_eth_dev_get_port_by_name() (struct rte_eth_dev.device->name). Without further intervention, the above suggestion won't work in DPDK 17.11. [1] http://dpdk.org/browse/dpdk/commit/?id=a1e7c17555e8f77d520ba5f06ed26c00e77a2bd1 > If necessary, the PCI address to Verbs device name conversion can be > performed like the PMD itself by looking at sysfs, e.g: > > $ PCI=0000:02:42.0 > $ ls /sys/bus/pci/devices/$PCI/infiniband/ > mlx4_0 > > Since DPDK 17.05, there's also a way to select what ports the mlx4 PMD > should register DPDK devices for by providing a "port" parameter (DPDK > devargs). The default is to enable them all, see: > > > http://dpdk.org/browse/dpdk/commit/?id=001a520e419fdbab8d83f572c2a4414c7bc8ed07 > > This can help for single port use cases where several ports are registered > by the PMD and the wrong one is used by the application. This workaround still stands though. -- Adrien Mazarguil 6WIND _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
