Hi Bhanu Prakash, I already check the BIOS, VT-d is enabled by default. From the dmesg, how can i find if VT-d is enabled, i see "IOMMU enabled", i don't understand the rest.
[root@localhost ~]# dmesg | grep -e DMAR -e IOMMU [ 0.000000] ACPI: DMAR 0x00000000BDDAD200 000558 (v01 HP ProLiant 00000001 \xffffffd2? 0000162E) [ 0.000000] DMAR: IOMMU enabled [ 0.069333] DMAR: Host address width 46 [ 0.069335] DMAR: DRHD base: 0x000000fbefe000 flags: 0x0 [ 0.069341] DMAR: dmar0: reg_base_addr fbefe000 ver 1:0 cap d2078c106f0466 ecap f020de [ 0.069342] DMAR: DRHD base: 0x000000f4ffe000 flags: 0x1 [ 0.069346] DMAR: dmar1: reg_base_addr f4ffe000 ver 1:0 cap d2078c106f0466 ecap f020de [ 0.069347] DMAR: RMRR base: 0x000000bdffd000 end: 0x000000bdffffff [ 0.069349] DMAR: RMRR base: 0x000000bdff6000 end: 0x000000bdffcfff [ 0.069349] DMAR: RMRR base: 0x000000bdf83000 end: 0x000000bdf84fff [ 0.069351] DMAR: RMRR base: 0x000000bdf7f000 end: 0x000000bdf82fff [ 0.069352] DMAR: RMRR base: 0x000000bdf6f000 end: 0x000000bdf7efff [ 0.069353] DMAR: RMRR base: 0x000000bdf6e000 end: 0x000000bdf6efff [ 0.069355] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff [ 0.069356] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff [ 0.069356] DMAR: RMRR base: 0x000000bddde000 end: 0x000000bdddefff [ 0.069357] DMAR: ATSR flags: 0x0 [ 0.069360] DMAR-IR: IOAPIC id 10 under DRHD base 0xfbefe000 IOMMU 0 [ 0.069361] DMAR-IR: IOAPIC id 8 under DRHD base 0xf4ffe000 IOMMU 1 [ 0.069362] DMAR-IR: IOAPIC id 0 under DRHD base 0xf4ffe000 IOMMU 1 [ 0.069362] DMAR-IR: HPET id 0 under DRHD base 0xf4ffe000 [ 0.069364] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [ 0.069364] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [ 0.070293] DMAR-IR: Enabled IRQ remapping in xapic mode [ 0.996061] DMAR: dmar0: Using Queued invalidation [ 0.996220] DMAR: dmar1: Using Queued invalidation [ 0.996477] DMAR: Hardware identity mapping for device 0000:00:00.0 [ 0.996479] DMAR: Hardware identity mapping for device 0000:00:01.0 [ 0.996481] DMAR: Hardware identity mapping for device 0000:00:01.1 [ 0.996483] DMAR: Hardware identity mapping for device 0000:00:02.0 [ 0.996484] DMAR: Hardware identity mapping for device 0000:00:02.1 [ 0.996489] DMAR: Hardware identity mapping for device 0000:00:02.2 [ 0.996490] DMAR: Hardware identity mapping for device 0000:00:02.3 [ 0.996492] DMAR: Hardware identity mapping for device 0000:00:03.0 [ 0.996494] DMAR: Hardware identity mapping for device 0000:00:03.1 [ 0.996495] DMAR: Hardware identity mapping for device 0000:00:03.2 [ 0.996497] DMAR: Hardware identity mapping for device 0000:00:03.3 [ 0.996499] DMAR: Hardware identity mapping for device 0000:00:04.0 [ 0.996501] DMAR: Hardware identity mapping for device 0000:00:04.1 [ 0.996502] DMAR: Hardware identity mapping for device 0000:00:04.2 [ 0.996504] DMAR: Hardware identity mapping for device 0000:00:04.3 [ 0.996505] DMAR: Hardware identity mapping for device 0000:00:04.4 [ 0.996507] DMAR: Hardware identity mapping for device 0000:00:04.5 [ 0.996509] DMAR: Hardware identity mapping for device 0000:00:04.6 [ 0.996510] DMAR: Hardware identity mapping for device 0000:00:04.7 [ 0.996512] DMAR: Hardware identity mapping for device 0000:00:05.0 [ 0.996514] DMAR: Hardware identity mapping for device 0000:00:05.2 [ 0.996515] DMAR: Hardware identity mapping for device 0000:00:05.4 [ 0.996517] DMAR: Hardware identity mapping for device 0000:00:11.0 [ 0.996519] DMAR: Hardware identity mapping for device 0000:00:1a.0 [ 0.996520] DMAR: Hardware identity mapping for device 0000:00:1c.0 [ 0.996522] DMAR: Hardware identity mapping for device 0000:00:1c.7 [ 0.996523] DMAR: Hardware identity mapping for device 0000:00:1d.0 [ 0.996525] DMAR: Hardware identity mapping for device 0000:00:1f.0 [ 0.996534] DMAR: Hardware identity mapping for device 0000:01:00.1 [ 0.996536] DMAR: Hardware identity mapping for device 0000:01:00.4 [ 0.996545] DMAR: Hardware identity mapping for device 0000:1f:08.0 [ 0.996547] DMAR: Hardware identity mapping for device 0000:1f:09.0 [ 0.996548] DMAR: Hardware identity mapping for device 0000:1f:0a.0 [ 0.996550] DMAR: Hardware identity mapping for device 0000:1f:0a.1 [ 0.996552] DMAR: Hardware identity mapping for device 0000:1f:0a.2 [ 0.996553] DMAR: Hardware identity mapping for device 0000:1f:0a.3 [ 0.996555] DMAR: Hardware identity mapping for device 0000:1f:0b.0 [ 0.996556] DMAR: Hardware identity mapping for device 0000:1f:0b.3 [ 0.996558] DMAR: Hardware identity mapping for device 0000:1f:0c.0 [ 0.996559] DMAR: Hardware identity mapping for device 0000:1f:0c.1 [ 0.996561] DMAR: Hardware identity mapping for device 0000:1f:0c.2 [ 0.996563] DMAR: Hardware identity mapping for device 0000:1f:0c.3 [ 0.996564] DMAR: Hardware identity mapping for device 0000:1f:0c.4 [ 0.996566] DMAR: Hardware identity mapping for device 0000:1f:0c.5 [ 0.996567] DMAR: Hardware identity mapping for device 0000:1f:0d.0 [ 0.996569] DMAR: Hardware identity mapping for device 0000:1f:0d.1 [ 0.996570] DMAR: Hardware identity mapping for device 0000:1f:0d.2 [ 0.996572] DMAR: Hardware identity mapping for device 0000:1f:0d.3 [ 0.996573] DMAR: Hardware identity mapping for device 0000:1f:0d.4 [ 0.996575] DMAR: Hardware identity mapping for device 0000:1f:0d.5 [ 0.996576] DMAR: Hardware identity mapping for device 0000:1f:0e.0 [ 0.996578] DMAR: Hardware identity mapping for device 0000:1f:0e.1 [ 0.996579] DMAR: Hardware identity mapping for device 0000:1f:0f.0 [ 0.996581] DMAR: Hardware identity mapping for device 0000:1f:0f.1 [ 0.996582] DMAR: Hardware identity mapping for device 0000:1f:0f.2 [ 0.996584] DMAR: Hardware identity mapping for device 0000:1f:0f.3 [ 0.996585] DMAR: Hardware identity mapping for device 0000:1f:0f.4 [ 0.996587] DMAR: Hardware identity mapping for device 0000:1f:0f.5 [ 0.996588] DMAR: Hardware identity mapping for device 0000:1f:10.0 [ 0.996590] DMAR: Hardware identity mapping for device 0000:1f:10.1 [ 0.996592] DMAR: Hardware identity mapping for device 0000:1f:10.2 [ 0.996593] DMAR: Hardware identity mapping for device 0000:1f:10.3 [ 0.996595] DMAR: Hardware identity mapping for device 0000:1f:10.4 [ 0.996596] DMAR: Hardware identity mapping for device 0000:1f:10.5 [ 0.996598] DMAR: Hardware identity mapping for device 0000:1f:10.6 [ 0.996600] DMAR: Hardware identity mapping for device 0000:1f:10.7 [ 0.996601] DMAR: Hardware identity mapping for device 0000:1f:13.0 [ 0.996603] DMAR: Hardware identity mapping for device 0000:1f:13.1 [ 0.996604] DMAR: Hardware identity mapping for device 0000:1f:13.4 [ 0.996607] DMAR: Hardware identity mapping for device 0000:1f:13.5 [ 0.996609] DMAR: Hardware identity mapping for device 0000:1f:16.0 [ 0.996611] DMAR: Hardware identity mapping for device 0000:1f:16.1 [ 0.996612] DMAR: Hardware identity mapping for device 0000:1f:16.2 [ 0.996614] DMAR: Hardware identity mapping for device 0000:1f:1c.0 [ 0.996616] DMAR: Hardware identity mapping for device 0000:1f:1c.1 [ 0.996617] DMAR: Hardware identity mapping for device 0000:1f:1d.0 [ 0.996619] DMAR: Hardware identity mapping for device 0000:1f:1d.1 [ 0.996621] DMAR: Hardware identity mapping for device 0000:1f:1d.2 [ 0.996622] DMAR: Hardware identity mapping for device 0000:1f:1d.3 [ 0.996624] DMAR: Hardware identity mapping for device 0000:1f:1d.4 [ 0.996625] DMAR: Hardware identity mapping for device 0000:1f:1d.5 [ 0.996627] DMAR: Hardware identity mapping for device 0000:1f:1e.0 [ 0.996629] DMAR: Hardware identity mapping for device 0000:1f:1e.1 [ 0.996631] DMAR: Hardware identity mapping for device 0000:1f:1e.2 [ 0.996632] DMAR: Hardware identity mapping for device 0000:1f:1e.3 [ 0.996634] DMAR: Hardware identity mapping for device 0000:1f:1e.4 [ 0.996635] DMAR: Hardware identity mapping for device 0000:1f:1e.5 [ 0.996637] DMAR: Hardware identity mapping for device 0000:1f:1e.6 [ 0.996639] DMAR: Hardware identity mapping for device 0000:1f:1e.7 [ 0.996647] DMAR: Hardware identity mapping for device 0000:20:00.0 [ 0.996649] DMAR: Hardware identity mapping for device 0000:20:01.0 [ 0.996651] DMAR: Hardware identity mapping for device 0000:20:01.1 [ 0.996652] DMAR: Hardware identity mapping for device 0000:20:02.0 [ 0.996797] DMAR: Setting RMRR: [ 0.996815] DMAR: Setting identity map for device 0000:01:00.0 [0xbddde000 - 0xbdddefff] [ 0.996853] DMAR: Setting identity map for device 0000:01:00.2 [0xbddde000 - 0xbdddefff] [ 0.996875] DMAR: Ignoring identity map for HW passthrough device 0000:01:00.4 [0xbddde000 - 0xbdddefff] [ 0.996890] DMAR: Setting identity map for device 0000:00:1f.2 [0xe8000 - 0xe8fff] [ 0.996911] DMAR: Setting identity map for device 0000:01:00.0 [0xe8000 - 0xe8fff] [ 0.996927] DMAR: Setting identity map for device 0000:01:00.2 [0xe8000 - 0xe8fff] [ 0.996955] DMAR: Setting identity map for device 0000:02:00.0 [0xe8000 - 0xe8fff] [ 0.996998] DMAR: Setting identity map for device 0000:03:00.0 [0xe8000 - 0xe8fff] [ 0.997043] DMAR: Setting identity map for device 0000:03:00.1 [0xe8000 - 0xe8fff] [ 0.997080] DMAR: Setting identity map for device 0000:03:00.2 [0xe8000 - 0xe8fff] [ 0.997121] DMAR: Setting identity map for device 0000:03:00.3 [0xe8000 - 0xe8fff] [ 0.997158] DMAR: Setting identity map for device 0000:04:00.0 [0xe8000 - 0xe8fff] [ 0.997202] DMAR: Setting identity map for device 0000:04:00.1 [0xe8000 - 0xe8fff] [ 0.997226] DMAR: Setting identity map for device 0000:00:1f.2 [0xf4000 - 0xf4fff] [ 0.997228] DMAR: Setting identity map for device 0000:01:00.0 [0xf4000 - 0xf4fff] [ 0.997231] DMAR: Setting identity map for device 0000:01:00.2 [0xf4000 - 0xf4fff] [ 0.997233] DMAR: Setting identity map for device 0000:02:00.0 [0xf4000 - 0xf4fff] [ 0.997235] DMAR: Setting identity map for device 0000:03:00.0 [0xf4000 - 0xf4fff] [ 0.997237] DMAR: Setting identity map for device 0000:03:00.1 [0xf4000 - 0xf4fff] [ 0.997239] DMAR: Setting identity map for device 0000:03:00.2 [0xf4000 - 0xf4fff] [ 0.997241] DMAR: Setting identity map for device 0000:03:00.3 [0xf4000 - 0xf4fff] [ 0.997243] DMAR: Setting identity map for device 0000:04:00.0 [0xf4000 - 0xf4fff] [ 0.997245] DMAR: Setting identity map for device 0000:04:00.1 [0xf4000 - 0xf4fff] [ 0.997246] DMAR: Setting identity map for device 0000:00:1f.2 [0xbdf6e000 - 0xbdf6efff] [ 0.997262] DMAR: Setting identity map for device 0000:01:00.0 [0xbdf6e000 - 0xbdf6efff] [ 0.997270] DMAR: Setting identity map for device 0000:01:00.2 [0xbdf6e000 - 0xbdf6efff] [ 0.997279] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6e000 - 0xbdf6efff] [ 0.997293] DMAR: Setting identity map for device 0000:03:00.0 [0xbdf6e000 - 0xbdf6efff] [ 0.997311] DMAR: Setting identity map for device 0000:03:00.1 [0xbdf6e000 - 0xbdf6efff] [ 0.997326] DMAR: Setting identity map for device 0000:03:00.2 [0xbdf6e000 - 0xbdf6efff] [ 0.997340] DMAR: Setting identity map for device 0000:03:00.3 [0xbdf6e000 - 0xbdf6efff] [ 0.997356] DMAR: Setting identity map for device 0000:04:00.0 [0xbdf6e000 - 0xbdf6efff] [ 0.997370] DMAR: Setting identity map for device 0000:04:00.1 [0xbdf6e000 - 0xbdf6efff] [ 0.997385] DMAR: Setting identity map for device 0000:00:1f.2 [0xbdf6f000 - 0xbdf7efff] [ 0.997388] DMAR: Setting identity map for device 0000:01:00.0 [0xbdf6f000 - 0xbdf7efff] [ 0.997390] DMAR: Setting identity map for device 0000:01:00.2 [0xbdf6f000 - 0xbdf7efff] [ 0.997393] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6f000 - 0xbdf7efff] [ 0.997395] DMAR: Setting identity map for device 0000:03:00.0 [0xbdf6f000 - 0xbdf7efff] [ 0.997397] DMAR: Setting identity map for device 0000:03:00.1 [0xbdf6f000 - 0xbdf7efff] [ 0.997399] DMAR: Setting identity map for device 0000:03:00.2 [0xbdf6f000 - 0xbdf7efff] [ 0.997402] DMAR: Setting identity map for device 0000:03:00.3 [0xbdf6f000 - 0xbdf7efff] [ 0.997404] DMAR: Setting identity map for device 0000:04:00.0 [0xbdf6f000 - 0xbdf7efff] [ 0.997406] DMAR: Setting identity map for device 0000:04:00.1 [0xbdf6f000 - 0xbdf7efff] [ 0.997409] DMAR: Setting identity map for device 0000:00:1f.2 [0xbdf7f000 - 0xbdf82fff] [ 0.997411] DMAR: Setting identity map for device 0000:01:00.0 [0xbdf7f000 - 0xbdf82fff] [ 0.997413] DMAR: Setting identity map for device 0000:01:00.2 [0xbdf7f000 - 0xbdf82fff] [ 0.997415] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf7f000 - 0xbdf82fff] [ 0.997417] DMAR: Setting identity map for device 0000:03:00.0 [0xbdf7f000 - 0xbdf82fff] [ 0.997419] DMAR: Setting identity map for device 0000:03:00.1 [0xbdf7f000 - 0xbdf82fff] [ 0.997423] DMAR: Setting identity map for device 0000:03:00.2 [0xbdf7f000 - 0xbdf82fff] [ 0.997425] DMAR: Setting identity map for device 0000:03:00.3 [0xbdf7f000 - 0xbdf82fff] [ 0.997427] DMAR: Setting identity map for device 0000:04:00.0 [0xbdf7f000 - 0xbdf82fff] [ 0.997430] DMAR: Setting identity map for device 0000:04:00.1 [0xbdf7f000 - 0xbdf82fff] [ 0.997432] DMAR: Setting identity map for device 0000:00:1f.2 [0xbdf83000 - 0xbdf84fff] [ 0.997434] DMAR: Setting identity map for device 0000:01:00.0 [0xbdf83000 - 0xbdf84fff] [ 0.997436] DMAR: Setting identity map for device 0000:01:00.2 [0xbdf83000 - 0xbdf84fff] [ 0.997438] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf83000 - 0xbdf84fff] [ 0.997440] DMAR: Setting identity map for device 0000:03:00.0 [0xbdf83000 - 0xbdf84fff] [ 0.997442] DMAR: Setting identity map for device 0000:03:00.1 [0xbdf83000 - 0xbdf84fff] [ 0.997443] DMAR: Setting identity map for device 0000:03:00.2 [0xbdf83000 - 0xbdf84fff] [ 0.997445] DMAR: Setting identity map for device 0000:03:00.3 [0xbdf83000 - 0xbdf84fff] [ 0.997447] DMAR: Setting identity map for device 0000:04:00.0 [0xbdf83000 - 0xbdf84fff] [ 0.997449] DMAR: Setting identity map for device 0000:04:00.1 [0xbdf83000 - 0xbdf84fff] [ 0.997451] DMAR: Setting identity map for device 0000:01:00.0 [0xbdff6000 - 0xbdffcfff] [ 0.997454] DMAR: Setting identity map for device 0000:01:00.2 [0xbdff6000 - 0xbdffcfff] [ 0.997456] DMAR: Ignoring identity map for HW passthrough device 0000:01:00.4 [0xbdff6000 - 0xbdffcfff] [ 0.997457] DMAR: Ignoring identity map for HW passthrough device 0000:00:1a.0 [0xbdffd000 - 0xbdffffff] [ 0.997458] DMAR: Ignoring identity map for HW passthrough device 0000:00:1d.0 [0xbdffd000 - 0xbdffffff] [ 0.997460] DMAR: Prepare 0-16MiB unity mapping for LPC [ 0.997461] DMAR: Ignoring identity map for HW passthrough device 0000:00:1f.0 [0x0 - 0xffffff] [ 0.997465] DMAR: Intel(R) Virtualization Technology for Directed I/O [ 1.351801] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping [ 1.362623] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping [ 1.373601] DMAR: 32bit 0000:01:00.4 uses non-identity mapping [ 297.035504] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. Regards kapil. On Tue, Aug 9, 2016 at 5:38 PM, Bodireddy, Bhanuprakash < bhanuprakash.bodire...@intel.com> wrote: > >-----Original Message----- > >From: discuss [mailto:discuss-boun...@openvswitch.org] On Behalf Of Kapil > >Adhikesavalu > >Sent: Tuesday, August 9, 2016 10:46 AM > >To: dev@openvswitch.org; disc...@openvswitch.org > >Subject: [ovs-discuss] OVS DPDK VFIO error > > > >Hi, > > > >On a Intel xeon E5-2697 chip with iommu turned on with Intel NIC 82599, i > am > >getting the following error while doing the NIC binding using VFIO. > >kernel: 4.23 fedora 23, i haven't tried the latest kernel yet. > > > >E5-2697 supports IOMMU VT-d > I hope you have already enabled VT-d in BIOS, can you check 'dmesg | grep > -e DMAR -e IOMMU'. > > > > >VFIO NIC binding steps, > >modprobe vfio-pci > >sudo /usr/bin/chmod a+x /dev/vfio > >sudo /usr/bin/chmod 0666 /dev/vfio/* > >$DPDK_DIR/tools/dpdk_nic_bind.py --bind=vfio-pci 0000:04:00.0 > >$DPDK_DIR/tools/dpdk_nic_bind.py --status > >Error > >===== > >EAL: Detected 48 lcore(s) > >EAL: Probing VFIO support... > >EAL: IOMMU type 1 (Type 1) is supported > >EAL: IOMMU type 8 (No-IOMMU) is not supported > >EAL: VFIO support initialized > > > >EAL: Master lcore 1 is ready (tid=83504bc0;cpuset=[1]) > >EAL: PCI device 0000:04:00.0 on NUMA socket 0 > >EAL: probe driver: 8086:154d rte_ixgbe_pmd > >EAL: set IOMMU type 1 (Type 1) failed, error 1 (Operation not permitted) > >EAL: set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) > >EAL: 0000:04:00.0 failed to select IOMMU type > >EAL: Error - exiting with code: 1 > > Cause: Requested device 0000:04:00.0 cannot be used > > > >dmesg: > >====== > >[ 0.997461] DMAR: Ignoring identity map for HW passthrough device > >0000:00:1f.0 [0x0 - 0xffffff] > >[ 0.997465] DMAR: Intel(R) Virtualization Technology for Directed I/O > >[ 1.351801] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping > >[ 1.362623] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping > >[ 1.373601] DMAR: 32bit 0000:01:00.4 uses non-identity mapping > >[ 297.035504] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU > domain > >attach due to platform RMRR requirement. Contact your platform vendor. > > > > > >[root@localhost bin]# cat /proc/cmdline > >BOOT_IMAGE=/vmlinuz-4.2.3-300.fc23.x86_64 root=/dev/mapper/fedora- > >root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet > >default_hugepagesz=1G hugepagesz=1G hugepages=16 hugepagesz=2M > >hugepages=2048 iommu=pt intel_iommu=on > > I don’t see any problem with your cmdline as iommu=pt and intel_iommu is > added. > > Regards, > Bhanu Prakash. > > > > >demsg | grep 10G - 82599 controller > >04:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 > Adapter > >(rev 01) > >04:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 > Adapter > >(rev 01) > > > >Regards > >Kapil. > _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
