Re: [RFC] kvmtool: add support for modern virtio-pci
Hi,
> That was indeed the ISR field. Fixing that makes seabios reach the same point
> as
> legacy virtio before failing.
>
> I don't see the original correspondence about seabios failures you've
> reported, if
> you want to forward them over we can look at it further.
It was a few months back, when I posted the seabios patches for kvmtool
to both seabios and kvm lists.
Issue #1 is that kvmtool adds a bunch of kernel command line options,
not only for stuff like rootfs configuration, but also to force specific
things the kernel fails to autodetect (or to speedup boot by
shortcutting hardware probing). Among them is "pci=conf1", without that
the kernel doesn't find a pci bus and therefore also doesn't find the
virtio-{blk,net} devices.
So, when booting with seabios and let grub or another boot loader load
the kernel from the guest disk image those kernel arguments are not
there. Of course you can boot the image with qemu, add "pci=conf1" to
grub.cfg (maybe others are required too, don't remember exactly), then
try again with kvmtool. That gets the boot one step further and leads
to ...
Issue #2: virtio kernel drivers fail initialize the virtio devices.
I suspect virtio device reset is not implemented properly and because of
that the state of the device as left by seabios confuses the kernel
driver. Didn't check that in detail though.
cheers,
Gerd
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
On 11/19/2015 02:21 AM, Gerd Hoffmann wrote: > On Mi, 2015-11-18 at 23:01 -0500, Sasha Levin wrote: >> On 11/18/2015 11:00 PM, Sasha Levin wrote: >>> Anyways, I debugged it for a bit a found that seabios attempts to write to >>> the notification BAR, I look further tomorrow to narrow it down and fix it. >> >> Err, *read*, obviously. >> >> I've never implemented that because the kernel doesn't try to do that (it >> doesn't >> make much sense, I think...). > > It doesn't make sense indeed (kvmtool still shouldn't segfault though), > and on a quick look I can't spot a place in seabios doing that ... > > It's reading ISR, as part of device reset, to make sure any pending > interrupts are cleared. That was indeed the ISR field. Fixing that makes seabios reach the same point as legacy virtio before failing. I don't see the original correspondence about seabios failures you've reported, if you want to forward them over we can look at it further. Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
On 11/18/2015 03:22 AM, Gerd Hoffmann wrote: > /me goes undust the kvmtool patches for seabios. > > (see https://www.kraxel.org/cgit/seabios/commit/?h=kvmtool, > build with CONFIG_KVMTOOL=y + CONFIG_DEBUG_LEVEL=9) > > nilsson kraxel ~# ~kraxel/projects/kvmtool/lkvm run --name seabios > --firmware /home/kraxel/projects/seabios/out-bios-kvmtool/bios.bin > --disk /vmdisk/cloud/persistent/Fedora-Cloud-Base-22-20150521.x86_64.raw > # lkvm run -k /boot/vmlinuz-3.10.0-324.el7.x86_64 -m 448 -c 4 --name > seabios Thanks for testing! I didn't even thing about seabios as a testing target. I tried to do what you described, and built seabios with: $ cat .config | grep 'KVMTOOL\|DEBUG' CONFIG_KVMTOOL=y CONFIG_DEBUG_LEVEL=9 But when booting, it just hangs on: $ ./lkvm run --firmware ~/seabios/out/bios.bin -d dummy # lkvm run -k /boot/vmlinuz-4.2.0-17-generic -m 448 -c 4 --name guest-12566 And same result with --virtio-legacy... What did I miss? Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
Hi, > Thanks for testing! I didn't even thing about seabios as a testing target. Not surprising, support isn't upstream, ran into a bunch of issues[1][2] last time I tried to combine the two, ran into some issues and nobody seemed to care, so the seabios patches where just sitting in a branch in my repo ... > $ cat .config | grep 'KVMTOOL\|DEBUG' > CONFIG_KVMTOOL=y > CONFIG_DEBUG_LEVEL=9 Hmm, 'CONFIG_KVMTOOL=y > .config; make olddefconfig' should give you a working configuration. Setting 'CONFIG_DEBUG_LEVEL=9' is useful for trouble-shooting as it makes the virtio drivers more verbose, but not mandatory to have. Serial line support is needed to get output: CONFIG_DEBUG_SERIAL=y CONFIG_DEBUG_SERIAL_PORT=0x3f8 Also I think rom size must be 128k b/c kvmtool expects it to be that way: CONFIG_ROM_SIZE=128 But those are the defaults, and after "make olddefconfig" you should already have them ... cheers, Gerd [1] kernel doesn't find pci (can be worked around by tweaking kernel command line in boot loader config). [2] kernel virtio drivers fail to initialize (probably device reset not working properly). -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
On Mi, 2015-11-18 at 00:11 -0500, Sasha Levin wrote: > This is a first go at adding support for the modern (based on the 1.0 virtio > spec) virtio-pci implementation. > To sum it up: this is a lightly tested version for feedback about the design > and to weed out major bugs people notice. Feedback is very welcome! /me goes undust the kvmtool patches for seabios. (see https://www.kraxel.org/cgit/seabios/commit/?h=kvmtool, build with CONFIG_KVMTOOL=y + CONFIG_DEBUG_LEVEL=9) nilsson kraxel ~# ~kraxel/projects/kvmtool/lkvm run --name seabios --firmware /home/kraxel/projects/seabios/out-bios-kvmtool/bios.bin --disk /vmdisk/cloud/persistent/Fedora-Cloud-Base-22-20150521.x86_64.raw # lkvm run -k /boot/vmlinuz-3.10.0-324.el7.x86_64 -m 448 -c 4 --name seabios Changing serial settings was 0/0 now 3/0 SeaBIOS (version rel-1.9.0-7-g532b527) BUILD: gcc: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4) binutils: version 2.23.52.0.1-55.el7 20130226 kvmtool: probed 448 MB RAM. Add to e820 map: 1c00 1 malloc preinit Add to e820 map: 000a 0005 -1 Add to e820 map: 000f 0001 2 Add to e820 map: 1bfc 0004 2 phys_alloc zone=0x000f78e8 size=14464 align=10 ret=1bfbc6f0 (detail=0x1bfbc6c0) Relocating init from 0x000f40a0 to 0x1bfbc6f0 (size 14464) malloc init init ivt init bda Add to e820 map: 0009fc00 0400 2 init bios32 init PNPBIOS table init keyboard init mouse init pic math cp init PCI probe phys_alloc zone=0x1bfbff38 size=32 align=10 ret=1bfbc640 (detail=0x1bfbc610) PCI device 00:00.0 (vd=1af4:1000 c=0200) phys_alloc zone=0x1bfbff38 size=32 align=10 ret=1bfbc5f0 (detail=0x1bfbc5c0) PCI device 00:01.0 (vd=1af4:1001 c=0180) Found 2 PCI devices (max PCI bus is 00) tsc calibrate start=71959316 end=71968721 diff=9405 CPU Mhz=5 init timer init lpt Found 2 lpt ports init serial Found 4 serial ports init virtio-blk found virtio-blk at 0:1 phys_alloc zone=0x1bfbff40 size=80 align=10 ret=f78d0 (detail=0x1bfbc590) pci dev 0:1 virtio cap at 0x4c type 1 bar 0 at 0xd2000800 off +0x [mmio] pci dev 0:1 virtio cap at 0x5c type 2 bar 0 at 0xd2000800 off +0x0080 [mmio] pci dev 0:1 virtio cap at 0x70 type 3 bar 0 at 0xd2000800 off +0x0100 [mmio] pci dev 0:1 virtio cap at 0x80 type 4 bar 0 at 0xd2000800 off +0x0180 [mmio] pci dev 0:1 using modern (1.0) virtio mode vp write d2000814 (1) <- 0x0 Segmentation fault With '--virtio-legacy' added seabios manages to load the kernel from disk. cheers, Gerd -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
On 11/18/2015 11:00 PM, Sasha Levin wrote: > Anyways, I debugged it for a bit a found that seabios attempts to write to > the notification BAR, I look further tomorrow to narrow it down and fix it. Err, *read*, obviously. I've never implemented that because the kernel doesn't try to do that (it doesn't make much sense, I think...). Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
On 11/18/2015 12:52 PM, Gerd Hoffmann wrote: > Hi, > >> Thanks for testing! I didn't even thing about seabios as a testing target. > > Not surprising, support isn't upstream, ran into a bunch of issues[1][2] > last time I tried to combine the two, ran into some issues and nobody > seemed to care, so the seabios patches where just sitting in a branch in > my repo ... > >> $ cat .config | grep 'KVMTOOL\|DEBUG' >> CONFIG_KVMTOOL=y >> CONFIG_DEBUG_LEVEL=9 > > Hmm, 'CONFIG_KVMTOOL=y > .config; make olddefconfig' should give you a > working configuration. > > Setting 'CONFIG_DEBUG_LEVEL=9' is useful for trouble-shooting as it > makes the virtio drivers more verbose, but not mandatory to have. > > Serial line support is needed to get output: > > CONFIG_DEBUG_SERIAL=y > CONFIG_DEBUG_SERIAL_PORT=0x3f8 > > Also I think rom size must be 128k b/c kvmtool expects it to be that > way: > > CONFIG_ROM_SIZE=128 > > But those are the defaults, and after "make olddefconfig" you should > already have them ... It was the ROM_SIZE one as it seems, it was set to 0 here. Anyways, I debugged it for a bit a found that seabios attempts to write to the notification BAR, I look further tomorrow to narrow it down and fix it. Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] kvmtool: add support for modern virtio-pci
On Mi, 2015-11-18 at 23:01 -0500, Sasha Levin wrote: > On 11/18/2015 11:00 PM, Sasha Levin wrote: > > Anyways, I debugged it for a bit a found that seabios attempts to write to > > the notification BAR, I look further tomorrow to narrow it down and fix it. > > Err, *read*, obviously. > > I've never implemented that because the kernel doesn't try to do that (it > doesn't > make much sense, I think...). It doesn't make sense indeed (kvmtool still shouldn't segfault though), and on a quick look I can't spot a place in seabios doing that ... It's reading ISR, as part of device reset, to make sure any pending interrupts are cleared. cheers, Gerd -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
