Thanks Stephen. It looks like my memory controller is in the same IOMMU group. I assume this means I won't be able to do this with this NIC?
-Alan On Mon, Aug 7, 2023 at 8:26 PM Stephen Hemminger <[email protected]> wrote: > > On Mon, 7 Aug 2023 12:40:21 -0700 > Stephen Hemminger <[email protected]> wrote: > > > On Sun, 6 Aug 2023 11:33:43 -0400 > > Alan Beadle <[email protected]> wrote: > > > > > Hi, > > > > > > I need some help getting DPDK working. I am running Ubuntu 20.04 with > > > a modified Linux 5.4 kernel, but I have also tried the stock Ubuntu > > > 5.15 kernel with the same results. > > > > > > Here is my NIC info from lspci: > > > 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (3) > > > I219-LM (rev 09) > > > > > > I built and installed DPDK from source, and applied the following boot > > > flags: "intel_iommu=on iommu=pt" > > > > > > After booting I did the following as root: > > > echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > > > ifconfig enp0s31f6 down > > > dpdk-devbind.py --bind=vfio-pci 0000:00:1f.6 > > > > > > All of this appeared to work. > > > > > > I tried running the "skeleton" example program and got the following > > > output: > > > sudo ./build/basicfwd > > > EAL: Detected CPU lcores: 16 > > > EAL: Detected NUMA nodes: 1 > > > EAL: Detected shared linkage of DPDK > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > > > EAL: Selected IOVA mode 'VA' > > > EAL: VFIO support initialized > > > EAL: 0000:00:1f.6 VFIO group is not viable! Not all devices in IOMMU > > > group bound to VFIO or unbound > > > EAL: Requested device 0000:00:1f.6 cannot be used > > > TELEMETRY: No legacy callbacks, legacy socket not created > > > EAL: Error - exiting with code: 1 > > > Cause: Error: number of ports must be even > > > > > > I'm not at all familiar with DPDK or VFIO. What might the problem be? > > > > > > -Alan > > > > IOMMU groups are when multiple PCI devices share the same channel > > in the IOMMU. The group is used to determine what mapping to use when > > device does DMA. Since this is a security thing, devices in same IOMMU > > group can not be shared between kernel and non-kernel usage. > > > > The IOMMU group is determined by wiring on the motherboard. > > Usually it is things like multiple Ethernet ports sharing the same group. > > But can be much more confused. > > > > The only option is to unbind all devices in the group before using > > one with DPDK. > > More info on IOMMU groups is in kernel documentation: > https://www.kernel.org/doc/html/latest/driver-api/vfio.html > > and in this article > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/sect-iommu-deep-dive
