The max-ram-below-4g was the key here. I ve added this in the machine option
-machine pc-i440fx-xenial,accel=kvm,max-ram-below-4g=2G And QEMU starts fine with the passthrough. Milton > On 10 May 2017, at 20:25, Miltos Hatzimihail <[email protected]> wrote: > >> >> On 06/05/2017 09:39, Miltos Hatzimihail wrote: >>> Dear All >>> >>> I ve got a system with a Xilinx FPGA on a PCI express slot and I am trying >>> to pass it through using QEMU with vfio-pci. Here?s my command line: >>> >>> qemu-system-x86_64 -smp 4,sockets=2,cores=1,threads=2 -machine >>> pc-i440fx-xenial,accel=kvm --cpu qemu64 -enable-kvm -vnc :1 -device >>> vfio-pci,host=04:01.0 .. .. .. -mem 2G >>> >>> The VM starts OK with men = 2G. If I try to assign more memory to this host >>> (say 4GB), then the VM doesn?t even boot to the BIOS (it says that ?This >>> guest has initialised the display ?yet?). The weird thing is that if I omit >>> the passthrough, the VM starts ok with 4GB. >>> >>> The PCI device is configured as follows: >>> >>> Region 0: Memory at c1100000 (32-bit, non-prefetchable) [disabled] >>> [size=128K] >>> Region 1: Memory at c0000000 (32-bit, non-prefetchable) [disabled] >>> [size=16M] >>> Region 2: Memory at 80000000 (32-bit, non-prefetchable) [disabled] >>> [size=1G] >>> [virtual] Expansion ROM at c1000000 [disabled] [size=1M] >>> >>> I tried to find an answer online but I had no luck. What am I missing? >>> >>> Regards, >>> Milton >>> >> Look at Region 2: It is located at physical (virtual physical?) address >> 2GB (0x80000000), thus it won't work in a machine with RAM directly >> above the 2GB mark. >> > FWIW this setup works OK if I use Xen (the passthrough works and the men I am > assigning to the guest is 14G). So I am trying to understand what’s different. > >> I don't know what would happen if that card was inserted in a physical >> machine with > 2GB RAM. Maybe there would be a "memory hole" from >> 0x80000000 to 0xC1120000, maybe it would get assigned different physical >> addresses. >> > > I looked at the monitor for stdio when I give 4G and I see this > > 1 address-space: memory > 2 0000000000000000-ffffffffffffffff (prio 0, RW): system > 3 0000000000000000-00000000bfffffff (prio 0, RW): alias ram-below-4g > @pc.ram 0000000000000000-00000000bfffffff > 4 0000000000000000-ffffffffffffffff (prio -1, RW): pci > 5 00000000000a0000-00000000000bffff (prio 1, RW): vga-lowmem > 6 00000000000c0000-00000000000dffff (prio 1, RW): pc.rom > 7 00000000000e0000-00000000000fffff (prio 1, R-): alias isa-bios > @pc.bios 0000000000020000-000000000003ffff > 8 00000000fffc0000-00000000ffffffff (prio 0, R-): pc.bios > 9 00000000000a0000-00000000000bffff (prio 1, RW): alias smram-region > @pci 00000000000a0000-00000000000bffff > > But when I give 2G I see > > 1 address-space: memory > 2 0000000000000000-ffffffffffffffff (prio 0, RW): system > 3 0000000000000000-000000007fffffff (prio 0, RW): alias ram-below-4g > @pc.ram 0000000000000000-000000007fffffff > 4 0000000000000000-ffffffffffffffff (prio -1, RW): pci > 5 00000000000a0000-00000000000affff (prio 2, RW): alias vga.chain4 > @vga.vram 0000000000000000-000000000000ffff > 6 00000000000a0000-00000000000bffff (prio 1, RW): vga-lowmem > 7 00000000000c0000-00000000000dffff (prio 1, RW): pc.rom > 8 00000000000e0000-00000000000fffff (prio 1, R-): alias isa-bios > @pc.bios 0000000000020000-000000000003ffff > 9 0000000080000000-00000000bfffffff (prio 1, RW): VFIO 0000:01:00.0 > BAR 2 > 10 0000000080000000-00000000bfffffff (prio 0, RW): VFIO 0000:01:00.0 > BAR 2 mmap > 11 00000000c0000000-00000000c0ffffff (prio 1, RW): VFIO 0000:01:00.0 > BAR 1 > 12 00000000c0000000-00000000c0ffffff (prio 0, RW): VFIO 0000:01:00.0 > BAR 1 mmap > 13 00000000c1040000-00000000c105ffff (prio 1, RW): VFIO 0000:01:00.0 > BAR 0 > 14 00000000c1040000-00000000c105ffff (prio 0, RW): VFIO 0000:01:00.0 > BAR 0 mmap > 15 00000000c1070000-00000000c1070fff (prio 1, RW): vga.mmio > 16 00000000c1070400-00000000c107041f (prio 0, RW): vga ioports > remapped > 17 00000000c1070500-00000000c1070515 (prio 0, RW): bochs dispi > interface > 18 00000000c1070600-00000000c1070607 (prio 0, RW): qemu extended regs > 19 00000000c1071000-00000000c1071fff (prio 1, RW): virtio-net-pci-msix > 20 00000000c1071000-00000000c107102f (prio 0, RW): msix-table > 21 00000000c1071800-00000000c1071807 (prio 0, RW): msix-pba > 22 00000000fd000000-00000000fdffffff (prio 1, RW): vga.vram > 23 00000000fffc0000-00000000ffffffff (prio 0, R-): pc.bios > 24 00000000000a0000-00000000000bffff (prio 1, RW): alias smram-region > @pci 00000000000a0000-00000000000bffff > > which is what you are saying but I see more stuff missing - like vga.vram or > vga.mmio.. Is there a setting I am missing when I run QEMU? > >> If your host machine where it is actually inserted has more than 2GB RAM, >> look at the RAM and PCI memory mappings logged in /var/log/dmesg and in >> the /proc/ and /sys/ file systems. >> > It does have 20GB but I can’t figure out what’s going on.. > >> Enjoy >> >> Jakob >> -- >> Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com >> <https://www.wisemo.com/> >> Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 >> This public discussion message is non-binding and may contain errors. >> WiseMo - Remote Service Management for PCs, Phones and Embedded >
