Thanks for your inputs. So, does this mean, when ports are bound to VFIO driver(which uses IOMMU), we can run 2 DPDK apps concurrently, only if the 2 apps are accessing ports on 2 different Network cards. In other words, when ports are bound to VFIO driver, if 2 DPDK apps are running concurrently on the same system, then we need 2 Network cards installed on that system. So, I am guessing with 2 Network cards, under /dev, they would enumerate as 2 distinct paths, for example-> /dev/vfio/1 and /dev/vfio/2. Is this a correct understanding, please let me know.
On Thu, May 20, 2021 at 1:54 PM Stephen Hemminger < step...@networkplumber.org> wrote: > On Thu, 20 May 2021 13:34:35 -0700 > Sudharshan Krishnakumar <sudhar.ckris...@gmail.com> wrote: > > > Hi All, > > > > Have a Qs related to-> access to different ports on the same NIC by 2 > DPDK > > applications. > > > > > > > > Setup: 4-Port 10 Gig NIC card with ports bound for DPDK usage. > > > > > > > > With intel’s igb_uio kernel driver (used for binding NIC’s ports) using > > dpdk-setup.sh, at the /dev level, I see 4 device paths created, > > > > /dev/uio0, /dev/uio1, /dev/ui02, /dev/uio3 > > > > > > > > And I was able to run 2 DPDK applications concurrently(by blacklisting > the > > ports in EAL parameters), and they were able to take ownership of > different > > ports on the same NIC. > > > > For example DPDK_APP1 can take access of ports->0,1 and DPDK_APP2 can > take > > access of ports->2,3. > > > > > > > > Currently, I am using another DPDK app that relies on binding to VFIO > > driver. > > > > > > > > So I am trying to switch to binding NIC’s 4 ports to VFIO driver. > > > > > > > > Enabled VTIO in BIOS and added intel_iommu=on in kernel boot-param in > GRUB. > > > > > > > > After binding all 4 ports to VFIO driver, using dpdk-setup,sh, at the > /dev > > level, I only see these 2 device paths created-> /dev/vfio/vfio(control > > channel) > > > > and /dev/vfio/1(essentially just 1 path). > > > > > > > > I am able to run one DPDK application, for example-> PacketGen tool. > > > > But not able to run 2 DPDK applications concurrently accessing different > > ports on the same NIC, due to DPDK application not being able to get > access > > to the ports. > > > > For example, if I launch DPDK_APP1 first, it can get access to the ports, > > but DPDK_APP2 is NOT able to get access, and vice-versa. > > > > Note that both apps are trying to access different ports on the same NIC, > > and also ports have been black-listed as part of EAL params to the DPDK > > apps. > > > > I know this is possible with igb_uio driver. > > > > > > > > Please let me know, if it is possible to do this when ports are bound to > > VFIO driver. > > > > Thanks, > > > > Sudha > > The issue is that the IOMMU can't map two different areas at once > to same device. You might be able to use DPDK primary/secondary process > model to do this. >
