Re: [Qemu-discuss] qemu-system-arm -M virt v3.0.0-rc4 Linux kernel virtio boots fails with VFS: Cannot open root device "vda"
On Sat, Aug 11, 2018 at 8:17 AM, Auger Eric wrote: > Hi Ciro, Nerijus, > > On 08/11/2018 01:43 AM, Ciro Santilli wrote: > > Bisected to 17ec075a651a3f9613429c2d97018fce459ed943 hw/arm/virt: Use > > 256MB ECAM region by default > > > > @Eric: do you know what might be the best fix for me? > > > > Just replacing `-M virt` with `-M virt-3.0` did not solve the problem. > > Please try to use -machine virt,highmem=off > Thanks to all, that solved the problem. > your 32b FW most likely does not support highmem ECAM. > > Thanks > > Eric > > > > On Fri, Aug 10, 2018 at 11:02 AM, Nerijus Baliunas > > mailto:neri...@users.sourceforge.net>> > > wrote: > > > > On Thu, 9 Aug 2018 22:59:27 +0100 Ciro Santilli > > mailto:ciro.santi...@gmail.com>> wrote: > > > > > Analogous commands work for x86_64 and aarch64 > > > > > > All worked on v2.12.0 with the same images. > > > > > Kernel boot error message: > > > VFS: Cannot open root device "vda" or unknown-block(0,0): error -6 > > > > I have just tried to boot my armv7l virt-3.0 VM, which has root on > > /dev/vda3, and it boots > > successfully with 3.0.0-rc4. One is configured as Direct kernel boot > > and has > > Kernel args: console=ttyAMA0 rw > > root=UUID=00d450fb-138b-4faa-b2d2-c6c79950fb56 rootwait > > (IIRC I had to add root=UUID= option about 2 years ago, it booted > > w/o it before), > > and another is UEFI. I use libvirt, command lines for both are: > > > > /usr/bin/qemu-system-arm -name guest=fedora28-arm,debug-threads=on > > -S -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-7-fedora28-arm/master-key.aes > > -machine virt-3.0,accel=tcg,usb=off,dump-guest-core=off -m 2048 > > -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid > > 1afe58e9-f943-4ffe-889c-a3a6a4687937 -display none -no-user-config > > -nodefaults -chardev > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-7- > fedora28-arm/monitor.sock,server,nowait > > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc > > -no-shutdown -boot strict=on -kernel > > /var/lib/libvirt/images/vmlinuz-4.17.12-200.fc28.armv7hl -initrd > > /var/lib/libvirt/images/initramfs-4.17.12-200.fc28.armv7hl.img > > -append console=ttyAMA0 rw > > root=UUID=00d450fb-138b-4faa-b2d2-c6c79950fb56 rootwait -device > > i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1 -device > > pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 -drive > > file=/var/lib/libvirt/images/Fedora-Server-armhfp-22-TC4- > sda.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 > > -device > > virtio-blk-device,scsi=off,drive=drive-virtio-disk0,id= > virtio-disk0,bootindex=1 > > -netdev tap,fd=26,id=hostnet0 -device > > virtio-net-device,netdev=hostnet0,id=net0,mac=52:54:00:f1:3b:de > > -chardev pty,id=charserial0 -serial chardev:charserial0 -msg > > timestamp=on > > > > /usr/bin/qemu-system-arm -name > > guest=fedora28-arm-uefi,debug-threads=on -S -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-8-fedora28-arm-uefi/master-key.aes > > -machine virt-3.0,accel=tcg,usb=off,dump-guest-core=off -cpu > > cortex-a15 -drive > > file=/usr/share/edk2.git/arm/QEMU_EFI-pflash.raw,if=pflash, > format=raw,unit=0,readonly=on > > -drive > > file=/var/lib/libvirt/qemu/nvram/fedora28-arm-uefi_VARS. > fd,if=pflash,format=raw,unit=1 > > -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid > > 1b721aaa-0d2b-48f8-9697-ccca9f361248 -display none -no-user-config > > -nodefaults -chardev > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8- > fedora28-arm-uefi/monitor.sock,server,nowait > > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc > > -no-shutdown -boot strict=on -device > > virtio-serial-device,id=virtio-serial0 -drive > > file=/var/lib/libvirt/images/Fedora-Server-armhfp-22-TC4- > sda.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 > > -device > > virtio-blk-device,scsi=off,drive=drive-virtio-disk0,id= > virtio-disk0,bootindex=1 > > -netdev tap,fd=26,id=hostnet0 -device > > virtio-net-device,netdev=hostnet0,id=net0,mac=52:54:00:f1:3b:de > > -chardev pty,id=charserial0 -serial chardev:charserial0 -chardev > > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/ > target/domain-8-fedora28-arm-uefi/org.qemu.guest_agent.0,server,nowait > > -device > > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=org.qemu.guest_agent.0 > > -msg timestamp=on > > > > Regards, > > Nerijus > > > > > > Thanks Nerijus, I will try to bisect the options. >
Re: [Qemu-discuss] Slow boot in QEMU with virtio-scsi disks
Hi. On 11.08.2018 14:23, Ming Lei wrote: Please test for-4.19/block: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=for-4.19/block This slow boot issue should have been fixed by the following commits: 1311326cf475 blk-mq: avoid to synchronize rcu inside blk_cleanup_queue() 97889f9ac24f blk-mq: remove synchronize_rcu() from blk_mq_del_queue_tag_set() 5815839b3ca1 blk-mq: introduce new lock for protecting hctx->dispatch_wait 2278d69f030f blk-mq: don't pass **hctx to blk_mq_mark_tag_wait() 8ab6bb9ee8d0 blk-mq: cleanup blk_mq_get_driver_tag() Indeed, I can confirm that these commits fix the issue. Thanks a lot. -- Oleksandr Natalenko (post-factum)
[Qemu-discuss] Efficacy of jitterentropy RNG on qemu-kvm Guests
Subject: Efficacy of jitterentropy RNG on qemu-kvm Guests To: qemu-discuss@nongnu.org, libvirt-us...@redhat.com, whonix-de...@whonix.org Hello. I'm a distro maintainer and was wondering about the efficacy of entropy daemons like haveged and jitterentropyd in qemu-kvm. One of the authors of haveged [0] pointed out if the hardware cycles counter is emulated and deterministic, and thus predictible. He therefore does not recommend using HAVEGE on those systems. Is this the case with KVM's counters? PS. I will be setting VM CPU settings to host-passthrough. Bonus: Also if anyone knows the answer to this question about Xen please let me know because its the other main platform we support and they don't have the luxury of virtio-rng in PVH mode. Thanks. [0] https://github.com/BetterCrypto/Applied-Crypto-Hardening/commit/cf7cef7a870c1b77089b1bd6209ded6525b5a4e0#commitcomment-23006392
Re: [Qemu-discuss] Slow boot in QEMU with virtio-scsi disks
On Sat, Aug 11, 2018 at 5:47 PM, Oleksandr Natalenko wrote: > Hi. > > I'd like to resurrect previous discussion [1] regarding slow kernel boot > inside QEMU with virtio-scsi disks attached and blk_mq enabled. > > Symptom: > > [2.830857] ata1: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002100 > irq 36 > [2.834559] ata2: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002180 > irq 36 > [2.837746] ata3: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002200 > irq 36 > [2.841861] ata4: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002280 > irq 36 > [2.847899] ata5: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002300 > irq 36 > [2.853229] ata6: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002380 > irq 36 > [3.172159] ata1: SATA link down (SStatus 0 SControl 300) > [3.183552] ata5: SATA link down (SStatus 0 SControl 300) > [3.189925] ata3: SATA link down (SStatus 0 SControl 300) > [3.196156] ata6: SATA link down (SStatus 0 SControl 300) > [3.201136] ata2: SATA link down (SStatus 0 SControl 300) > [3.208559] ata4: SATA link down (SStatus 0 SControl 300) > [ 16.480972] sd 0:0:1:0: Power-on or device reset occurred > [ 16.481591] sd 0:0:0:0: [sda] 16777216 512-byte logical blocks: (8.59 > GB/8.00 GiB) > [ 16.481671] sd 0:0:0:0: [sda] Write Protect is off > [ 16.481815] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, > doesn't support DPO or FUA > [ 16.491325] sda: sda1 sda2 > [ 16.517532] sd 0:0:1:0: [sdb] 16777216 512-byte logical blocks: (8.59 > GB/8.00 GiB) > [ 16.525131] sr 0:0:2:0: Power-on or device reset occurred > [ 16.525974] sd 0:0:1:0: [sdb] Write Protect is off > [ 16.530946] sr 0:0:2:0: [sr0] scsi3-mmc drive: 16x/50x cd/rw xa/form2 > cdda tray > [ 16.543592] cdrom: Uniform CD-ROM driver Revision: 3.20 > [ 16.549815] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: enabled, > doesn't support DPO or FUA > [ 16.549833] sd 0:0:0:0: [sda] Attached SCSI disk > [ 16.572055] sdb: sdb1 sdb2 > [ 16.580463] sd 0:0:1:0: [sdb] Attached SCSI disk > > (note the hang that lasts for 13 seconds) > > The disks are attached to the VM in the following manner: > > -device virtio-scsi,id=scsi -device scsi-hd,drive=hd1 -drive > if=none,media=disk,id=hd1,file=sda.img,format=raw > > What I've tested so far: > > * 4.14.62 + virtio-scsi +blk_mq == slow boot > * 4.14.62 + virtio-scsi + no blk_mq == fast boot > * 4.17.13 + virtio-scsi +blk_mq == slow boot > * 4.18-rc8 + virtio-scsi +blk_mq == slow boot > > QEMU is of v2.12.1, runs with "-machine q35,accel=kvm -cpu host". Also, if > virtio-scsi disks are replaced with SATA disks, the hang does not occur > (although, QEMU has other issues with SATA, but that's another story [3]). > > Apparently, the commit that was mentioned in [2], > b5b6e8c8d3b4cbeb447a0f10c7d5de3caa573299, forces blk_mq for virtio_scsi, so > it cannot be disabled for new kernels. > > Any hint on how to avoid this hang while still having virtio-scsi disks and > blk_mq enabled please? Please test for-4.19/block: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=for-4.19/block This slow boot issue should have been fixed by the following commits: 1311326cf475 blk-mq: avoid to synchronize rcu inside blk_cleanup_queue() 97889f9ac24f blk-mq: remove synchronize_rcu() from blk_mq_del_queue_tag_set() 5815839b3ca1 blk-mq: introduce new lock for protecting hctx->dispatch_wait 2278d69f030f blk-mq: don't pass **hctx to blk_mq_mark_tag_wait() 8ab6bb9ee8d0 blk-mq: cleanup blk_mq_get_driver_tag() Thanks, Ming Lei
[Qemu-discuss] Slow boot in QEMU with virtio-scsi disks
Hi. I'd like to resurrect previous discussion [1] regarding slow kernel boot inside QEMU with virtio-scsi disks attached and blk_mq enabled. Symptom: [2.830857] ata1: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002100 irq 36 [2.834559] ata2: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002180 irq 36 [2.837746] ata3: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002200 irq 36 [2.841861] ata4: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002280 irq 36 [2.847899] ata5: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002300 irq 36 [2.853229] ata6: SATA max UDMA/133 abar m4096@0x98002000 port 0x98002380 irq 36 [3.172159] ata1: SATA link down (SStatus 0 SControl 300) [3.183552] ata5: SATA link down (SStatus 0 SControl 300) [3.189925] ata3: SATA link down (SStatus 0 SControl 300) [3.196156] ata6: SATA link down (SStatus 0 SControl 300) [3.201136] ata2: SATA link down (SStatus 0 SControl 300) [3.208559] ata4: SATA link down (SStatus 0 SControl 300) [ 16.480972] sd 0:0:1:0: Power-on or device reset occurred [ 16.481591] sd 0:0:0:0: [sda] 16777216 512-byte logical blocks: (8.59 GB/8.00 GiB) [ 16.481671] sd 0:0:0:0: [sda] Write Protect is off [ 16.481815] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 16.491325] sda: sda1 sda2 [ 16.517532] sd 0:0:1:0: [sdb] 16777216 512-byte logical blocks: (8.59 GB/8.00 GiB) [ 16.525131] sr 0:0:2:0: Power-on or device reset occurred [ 16.525974] sd 0:0:1:0: [sdb] Write Protect is off [ 16.530946] sr 0:0:2:0: [sr0] scsi3-mmc drive: 16x/50x cd/rw xa/form2 cdda tray [ 16.543592] cdrom: Uniform CD-ROM driver Revision: 3.20 [ 16.549815] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 16.549833] sd 0:0:0:0: [sda] Attached SCSI disk [ 16.572055] sdb: sdb1 sdb2 [ 16.580463] sd 0:0:1:0: [sdb] Attached SCSI disk (note the hang that lasts for 13 seconds) The disks are attached to the VM in the following manner: -device virtio-scsi,id=scsi -device scsi-hd,drive=hd1 -drive if=none,media=disk,id=hd1,file=sda.img,format=raw What I've tested so far: * 4.14.62 + virtio-scsi +blk_mq == slow boot * 4.14.62 + virtio-scsi + no blk_mq == fast boot * 4.17.13 + virtio-scsi +blk_mq == slow boot * 4.18-rc8 + virtio-scsi +blk_mq == slow boot QEMU is of v2.12.1, runs with "-machine q35,accel=kvm -cpu host". Also, if virtio-scsi disks are replaced with SATA disks, the hang does not occur (although, QEMU has other issues with SATA, but that's another story [3]). Apparently, the commit that was mentioned in [2], b5b6e8c8d3b4cbeb447a0f10c7d5de3caa573299, forces blk_mq for virtio_scsi, so it cannot be disabled for new kernels. Any hint on how to avoid this hang while still having virtio-scsi disks and blk_mq enabled please? Thanks. -- Oleksandr Natalenko (post-factum) [1] https://lists.gnu.org/archive/html/qemu-discuss/2018-07/msg00022.html [2] https://lists.gnu.org/archive/html/qemu-discuss/2018-07/msg00037.html [3] https://lists.nongnu.org/archive/html/qemu-devel/2018-05/msg06942.html
Re: [Qemu-discuss] qemu-system-arm -M virt v3.0.0-rc4 Linux kernel virtio boots fails with VFS: Cannot open root device "vda"
Hi Ciro, Nerijus, On 08/11/2018 01:43 AM, Ciro Santilli wrote: > Bisected to 17ec075a651a3f9613429c2d97018fce459ed943 hw/arm/virt: Use > 256MB ECAM region by default > > @Eric: do you know what might be the best fix for me? > > Just replacing `-M virt` with `-M virt-3.0` did not solve the problem. Please try to use -machine virt,highmem=off your 32b FW most likely does not support highmem ECAM. Thanks Eric > > On Fri, Aug 10, 2018 at 11:02 AM, Nerijus Baliunas > mailto:neri...@users.sourceforge.net>> > wrote: > > On Thu, 9 Aug 2018 22:59:27 +0100 Ciro Santilli > mailto:ciro.santi...@gmail.com>> wrote: > > > Analogous commands work for x86_64 and aarch64 > > > > All worked on v2.12.0 with the same images. > > > Kernel boot error message: > > VFS: Cannot open root device "vda" or unknown-block(0,0): error -6 > > I have just tried to boot my armv7l virt-3.0 VM, which has root on > /dev/vda3, and it boots > successfully with 3.0.0-rc4. One is configured as Direct kernel boot > and has > Kernel args: console=ttyAMA0 rw > root=UUID=00d450fb-138b-4faa-b2d2-c6c79950fb56 rootwait > (IIRC I had to add root=UUID= option about 2 years ago, it booted > w/o it before), > and another is UEFI. I use libvirt, command lines for both are: > > /usr/bin/qemu-system-arm -name guest=fedora28-arm,debug-threads=on > -S -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-7-fedora28-arm/master-key.aes > -machine virt-3.0,accel=tcg,usb=off,dump-guest-core=off -m 2048 > -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid > 1afe58e9-f943-4ffe-889c-a3a6a4687937 -display none -no-user-config > -nodefaults -chardev > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-7-fedora28-arm/monitor.sock,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc > -no-shutdown -boot strict=on -kernel > /var/lib/libvirt/images/vmlinuz-4.17.12-200.fc28.armv7hl -initrd > /var/lib/libvirt/images/initramfs-4.17.12-200.fc28.armv7hl.img > -append console=ttyAMA0 rw > root=UUID=00d450fb-138b-4faa-b2d2-c6c79950fb56 rootwait -device > i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1 -device > pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 -drive > > file=/var/lib/libvirt/images/Fedora-Server-armhfp-22-TC4-sda.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 > -device > > virtio-blk-device,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > -netdev tap,fd=26,id=hostnet0 -device > virtio-net-device,netdev=hostnet0,id=net0,mac=52:54:00:f1:3b:de > -chardev pty,id=charserial0 -serial chardev:charserial0 -msg > timestamp=on > > /usr/bin/qemu-system-arm -name > guest=fedora28-arm-uefi,debug-threads=on -S -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-fedora28-arm-uefi/master-key.aes > -machine virt-3.0,accel=tcg,usb=off,dump-guest-core=off -cpu > cortex-a15 -drive > > file=/usr/share/edk2.git/arm/QEMU_EFI-pflash.raw,if=pflash,format=raw,unit=0,readonly=on > -drive > > file=/var/lib/libvirt/qemu/nvram/fedora28-arm-uefi_VARS.fd,if=pflash,format=raw,unit=1 > -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid > 1b721aaa-0d2b-48f8-9697-ccca9f361248 -display none -no-user-config > -nodefaults -chardev > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-fedora28-arm-uefi/monitor.sock,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc > -no-shutdown -boot strict=on -device > virtio-serial-device,id=virtio-serial0 -drive > > file=/var/lib/libvirt/images/Fedora-Server-armhfp-22-TC4-sda.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 > -device > > virtio-blk-device,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > -netdev tap,fd=26,id=hostnet0 -device > virtio-net-device,netdev=hostnet0,id=net0,mac=52:54:00:f1:3b:de > -chardev pty,id=charserial0 -serial chardev:charserial0 -chardev > > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-8-fedora28-arm-uefi/org.qemu.guest_agent.0,server,nowait > -device > > virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 > -msg timestamp=on > > Regards, > Nerijus > > > Thanks Nerijus, I will try to bisect the options.
