Thank you for the prompt reply, Peter. Based on your recommendations, I downloaded the HEAD of QEMU's master branch a few hours ago and re-ran the test below with the "virt" machine model. After booting, I checked dmesg's KVM messages:
[ 10.645543] kvm [1]: HYP mode not available In addition, I see that /dev/kvm doesn't exist. I also tried the supported Xilinx machine models, and the only one that successfully booted and initialized KVM's hypervisor was "xlnx-versal-virt". That's the only reason I had used it below- I have no other preference for it. After booting QEMU with xlnx-versal-virt, I again tried setting up the guest QEMU instance. I didn't have time to create a new rootfs with QEMU 4 installed, so for the time being, I again used QEMU 3.1. If I use the "virt" model to boot the guest QEMU, I get the following error: qemu-system-aarch64: kvm_init_vcpu failed: Invalid argument In fact, I see this error with "xlnx-zcu102" and "xilinx-zynq-a9" as well. If I use "xlnx-versal-virt", then I again see the following error: ... /aarch64-poky-linux/qemu/3.1.0-r0/qemu-3.1.0/exec.c:892: cpu_address_space_init: Assertion `asidx == 0 || !kvm_enabled()' failed. which you already explained below. Do you happen to know of a specific configuration that properly boots a guest QEMU on top of a host QEMU using KVM? Any other thoughts are welcome as well. Thanks again, Kaushik On 2/5/19, 10:05 AM, "Peter Maydell" <[email protected]> wrote: On Tue, 5 Feb 2019 at 13:55, Kaushik Datta <[email protected]> wrote: > I am trying to show that KVM is working by booting a guest QEMU instance within a host QEMU instance using KVM. To provide context, the host QEMU is translating x86 to aarch64, while the guest QEMU is translating aarch 64 to aarch64, so is a candidate for KVM acceleration. I’m using QEMU 3.1.0 with the xlnx-versal-virt machine model for both the host and the guest since this is the only QEMU version + machine model combination that I’ve found that successfully initializes the KVM hypervisor (shown below). Personally I would recommend the "virt" board. That's the most common one people use to run KVM for aarch64 setups (and in general to run Linux in emulation too). I can't help so much with the xlnx-versal-virt board, but I would suggest not using that unless you specifically need to work with an emulation of Xilinx hardware. Emulation of the virtualization extensions in QEMU should work, but it's possible you're running into bugs in it, since it's not all that extensively tested. > In addition, both the host and the guest are using a Debian 4.9.130-2 kernel and a Yocto-generated rootfs. Here is the command I use to launch the host QEMU instance: > > qemu-system-aarch64 -M xlnx-versal-virt -m 2048 -cpu cortex-a53 -kernel vmlinuz-4.9.0-8-arm64 -initrd new_rootfs.cpio.gz -nographic -D "./qemu_debug.txt" > > After the host QEMU boots, here is the dmesg output regarding KVM: > > [ 12.058077] kvm [1]: HYP VA range: 800000000000:ffffffffffff > [ 12.061797] kvm [1]: Hyp mode initialized successfully > > In addition, /dev/kvm exists and I’m operating as root, so that’s good. However, when I do lsmod, I get an empty output, which is a concern. I don't think KVM is a module for aarch64, so this is expected. > Anyway, this is the command I used to boot the guest QEMU instance with KVM acceleration within the host QEMU: > > qemu-system-aarch64 -M xlnx-versal-virt -m 1024 -cpu cortex-a53 -kernel vmlinuz-4.9.0-8-arm64 -initrd new_rootfs.cpio.gz -accel kvm -enable-kvm -nographic -D "./qemu_debug.txt" > > However, this is the error that I get: > > qemu-system-aarch64: /home/kdatta/hpsc-bsp/HEAD/work/poky_build/tmp/work/aarch64-poky-linux/qemu/3.1.0-r0/qemu-3.1.0/exec.c:892: cpu_address_space_init: Assertion `asidx == 0 || !kvm_enabled()' failed. The error handling here is not great, but the problem I think that you're running into is that you're trying to run a setup where the guest CPU has the EL3 (TrustZone) feature enabled: this can't be emulated with KVM. The xlnx-versal-virt board always has this feature enabled, because that's what the Xilinx hardware setup is. If you switch to using the "virt" board instead (for at least the inner guest, but switching for the outer guest too would be sensible) this should go away, since 'virt' defaults to not enabling EL3 for the guest CPU. I would also suggest upgrading to a newer version of QEMU -- the command line there suggests you're using 3.1.0-rc0, which is a release candidate version of a version of QEMU that is not the most recent one. Using an RC for anything other than "testing before the release happens" isn't something I'd recommend. I also did fix some bugs in emulation of the virtualization extensions recently. Using 4.0 should be OK. thanks -- PMM
