On Tue, Feb 13, 2018 at 6:31 AM, Burakov, Anatoly <anatoly.bura...@intel.com
> On 12-Feb-18 10:00 PM, Ravi Kerur wrote:
>> Let me just give you what has been tested and working/nonworking
>> scenarios. Some of your questions might get answered as well.
>> Test bed is very simple with 2 VF's created under IXGBE PF on
>> host with one VF interface added to ovs-bridge on host and
>> another VF interface given to guest. Test connectivity between
>> VF's via ping.
>> Host and guest -- Kernel 4.9
>> Host -- Qemu 2.11.50 (tried both released 2.11 and tip of the
>> git (2.11.50))
>> DPDK -- 17.05.1 on host and guest
>> Host and guest -- booted with GRUB intel_iommu=on (which enables
>> IOMMU). Have tried with "iommu=on and intel_iommu=on" as well,
>> but iommu=on is not needed when intel_iommu=on is set.
>> Test-scenario-1: Host -- ixgbe_vf driver, Guest ixgbe_vf driver
>> ping works
>> Test-scenario-2: Host -- DPDK vfio-pci driver, Guest ixgbe_vf
>> driver ping works
>> Test-scenario-3: Host -- DPDK vfio-pci driver, Guest DPDK
>> vfio-pci driver, DMAR errors seen on host, ping doesn't work
>> OK, that makes it clearer, thanks. Does the third scenario work in
>> other DPDK versions?
>> No. Tried 16.11 same issue on guest and works fine on host.
>> So now we've moved from "this worked on 16.11" to "this never worked".
> It would be good to see output of rte_dump_physmem_layout() on both host
> and guest, and check which address triggers the DMAR error (i.e. if the
> physical address is present in mappings for either DPDK process).
Earlier I was focusing only on DMAR errors and I might have said 'it
worked' when I didn't notice them on host when dpdk was started on guest.
When trying to send packets out of that interface from guest I did see DMAR
errors. I am attaching information you requested. I have enabled
log-level=8 and files contain dpdk EAL/PMD logs as well.
on host, DMAR fault address from dmesg
[351576.998109] DMAR: DRHD: handling fault status reg 702
[351576.998113] DMAR: [DMA Read] Request device [04:10.0] fault addr
257617000 [fault reason 06] PTE Read access is not set
on guest (dump phys_mem_layout)
Segment 235: phys:0x257600000, len:2097152, virt:0x7fce87e00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0
PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fce87e0f4c0
sw_sc_ring=0x7fce87e07380 hw_ring=0x7fce87e17600 dma_addr=0x257617600
PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fce89c67d40
sw_sc_ring=0x7fce89c5fc00 hw_ring=0x7fce89c6fe80 dma_addr=0x25406fe80