Re: Announcing qboot, a minimal x86 firmware for QEMU
On Tue, May 26, 2015 at 9:47 AM, Stefan Hajnoczi stefa...@gmail.com wrote: On Fri, May 22, 2015 at 10:53:54AM +0800, Yong Wang wrote: On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote: On the QEMU side, there is no support yet for persistent memory and the NFIT tables from ACPI 6.0. Once that (and ACPI support) is added, qboot will automatically start using it. We are working on adding NFIT support into virtual bios. Great. I asked about this on the #pmem (irc.oftc.net) IRC channel last week. Which virtual bios are you targeting? Ping? Interest in persistent memory is picking up and I'd like to avoid duplicating work. Which pieces do you have patches for? 1. QEMU -device pmem,file=/path/to/dax/file,id=pmem1 and fw_cfg/ACPI info that gets passed to the guest 2. SeaBIOS NFIT ACPI table 3. ACPI NVDIMM DSM (probably not much needed, most features would be disabled) Thanks, Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 05/27/2015 12:30 PM, Paolo Bonzini wrote: On 26/05/2015 23:25, Christopher Covington wrote: On 05/25/2015 08:53 AM, Paolo Bonzini wrote: On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Could VirtIO specified replacements make sense for these peripherals? Not really. virtio is too heavyweight and you'd be reinventing the wheel unnecessarily. For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. The RTC can be replaced by kvmclock, the keyboard by virtio-console. Maybe we can provide an msr- or pci- based interface to fw_cfg. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On 27/05/2015 11:36, Avi Kivity wrote: On 05/27/2015 12:30 PM, Paolo Bonzini wrote: On 26/05/2015 23:25, Christopher Covington wrote: On 05/25/2015 08:53 AM, Paolo Bonzini wrote: On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Could VirtIO specified replacements make sense for these peripherals? Not really. virtio is too heavyweight and you'd be reinventing the wheel unnecessarily. For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. The RTC can be replaced by kvmclock, the keyboard by virtio-console. Maybe we can provide an msr- or pci- based interface to fw_cfg. The RTC is used for more than clock unfortunately. S3 support uses it for example, both to tell the firmware that it's an S3 resume and for resuming when the alarm fires. All in all, getting rid of ISA seems like chasing windmills. If you want to do that, fine, but then do not do that on a minimal firmware like qboot or SeaBIOS. For example, UEFI provides run-time services that let you access some low-level devices like this, and that's part of why the ARM virtual machine image specification mandates UEFI support. But even then, the ARM virtual machine image specification lets you choose between pl031 and a Xen pv console, and doesn't specify virtio-console... Paolo Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 26/05/2015 23:25, Christopher Covington wrote: On 05/25/2015 08:53 AM, Paolo Bonzini wrote: On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Could VirtIO specified replacements make sense for these peripherals? Not really. virtio is too heavyweight and you'd be reinventing the wheel unnecessarily. For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 27/05/2015 13:54, Peter Maydell wrote: On 27 May 2015 at 10:30, Paolo Bonzini pbonz...@redhat.com wrote: For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. Partly we do that because there were a number of reports that trying to use virtio for the console didn't work reliably... Using the stock UART that is widely supported in UEFI/uboot/kernel was a conservative design choice. The next thing that's likely to appear in virt is a PL061 GPIO device, which you need for CPU hotplug and external-shutdown-request notifications. Indeed, and the x86 Q35 chipset puts the ACPI registers... in the ISA/LPC bridge. :) Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 27 May 2015 at 10:30, Paolo Bonzini pbonz...@redhat.com wrote: For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. Partly we do that because there were a number of reports that trying to use virtio for the console didn't work reliably... Using the stock UART that is widely supported in UEFI/uboot/kernel was a conservative design choice. The next thing that's likely to appear in virt is a PL061 GPIO device, which you need for CPU hotplug and external-shutdown-request notifications. -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 05/27/2015 05:30 AM, Paolo Bonzini wrote: On 26/05/2015 23:25, Christopher Covington wrote: On 05/25/2015 08:53 AM, Paolo Bonzini wrote: On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Could VirtIO specified replacements make sense for these peripherals? Not really. virtio is too heavyweight I'd be curious to read where in your estimation this weight lies. Is it one-time initialization or recurring? Is it specific to the PCI transport or does MMIO suffer from it as well? and you'd be reinventing the wheel unnecessarily. In my mind the utility of peripherals that are instruction set architecture agnostic and can work over several transports is in reducing the amount of (emulator/hypervisor, firmware, and OS) code used, and therefore in need of maintenance, for common system emulation and virtualization use cases. For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. (UART is PL011; RTC is PL031) Thanks, Chris -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 27/05/2015 14:50, Christopher Covington wrote: Not really. virtio is too heavyweight I'd be curious to read where in your estimation this weight lies. Is it one-time initialization or recurring? Is it specific to the PCI transport or does MMIO suffer from it as well? It's heavyweight in the sense that virtio requires you to design a communication mechanism based on ring buffers. It's much harder than a few ad-hoc registers. And I know everyone is upset about the attack surface of QEMU these days, but effort would be much better spent adding QEMU-specific customizations to a static analysis tool (that e.g. would derive bound checks for the MemoryRegion read/write ops and be able to prove that said ops cannot access arrays out of their bounds). and you'd be reinventing the wheel unnecessarily. In my mind the utility of peripherals that are instruction set architecture agnostic and can work over several transports is in reducing the amount of (emulator/hypervisor, firmware, and OS) code used, and therefore in need of maintenance, for common system emulation and virtualization use cases. A fully processor-agnostic hardware architecture is a non-goal. You'll always have stuff like interrupt controllers that is extremely tied to the processor. If you want to abstract hardware, use the firmware, Luke! Things such as UEFI and ACPI are there for exactly this reason. We will be able to reuse a lot of x86 hotplug code on ARM using ACPI. And if you don't want to use ACPI, you can always write native OS drivers for the same hotplug hardware. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
* Paolo Bonzini (pbonz...@redhat.com) wrote: On 26/05/2015 23:25, Christopher Covington wrote: On 05/25/2015 08:53 AM, Paolo Bonzini wrote: On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Could VirtIO specified replacements make sense for these peripherals? Not really. virtio is too heavyweight and you'd be reinventing the wheel unnecessarily. I see reasons to replace some but not all these components; and there's no point in replacing the ISA/LPC bridge since it's got nothing at all in it. For example, ARM's -M virt uses a pl011 block for the RTC, and also uses fw_cfg. Another commonly used ISA device is the UART, for which again -M virt uses a pl031. I don't see much point in replacing the simple PC uart with anything virtio; I can imagine that you might want to go down to something really trivial with non of the bells and whistles; but a UART is pretty simple. The PC RTC though, it's a bit of a disaster that's had 30 years of random cruft added into it to hold random things that should never have been there. Dave Paolo -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On Fri, May 22, 2015 at 10:53:54AM +0800, Yong Wang wrote: On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote: On the QEMU side, there is no support yet for persistent memory and the NFIT tables from ACPI 6.0. Once that (and ACPI support) is added, qboot will automatically start using it. We are working on adding NFIT support into virtual bios. Great. I asked about this on the #pmem (irc.oftc.net) IRC channel last week. Which virtual bios are you targeting? Stefan pgp1LQizfjfkd.pgp Description: PGP signature
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 05/25/2015 08:53 AM, Paolo Bonzini wrote: On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Could VirtIO specified replacements make sense for these peripherals? Chris -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
2015-05-23 6:55 GMT+03:00 Kevin O'Connor ke...@koconnor.net: Out of curiosity, I ran some additional timing tests. With SeaBIOS fully stripped down (via Kconfig), it takes ~20ms to get to the boot phase on my old AMD system. Of that 20ms, ~7ms is to enable shadow ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci init, and ~6ms is to make the shadow ram area read-only. The time in the remaining parts of the SeaBIOS code is so small that it's hard to measure. Can you share config for seabios? As i understand i can safety to remove keybord, ps2, usb, ata/ahci and leave only virtio (in case of using qemu to boo linux/freebsd/windows systems) ? -- Vasiliy Tolstov, e-mail: v.tols...@selfip.ru -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On 23/05/2015 01:23, Kevin O'Connor wrote: Hi Paolo, I'm curious if you've tried profiling SeaBIOS to see where it is spending unnecessary time? No, I really wanted to get the absolute minimum time needed to get to the kernel. I announced it publicly because I think it's also instructive. I wonder if a stripped down SeaBIOS could obtain sufficient performance. It most likely could. However, some of the features (e.g. booting vmlinuz from parallel flash or, in the future, from persistent memory) probably do not fit SeaBIOS very well. One would have to redo them otherwise, for example using a DMA interface in fw_cfg. Paolo The page at http://seabios.org/Debugging#Timing_debug_messages describes how to do basic profiling via timing of debug messages. The default SeaBIOS build takes ~180ms on my (old AMD) system. But, by removing drivers and options via Kconfig I was able to bring it down to ~25ms. I suspect some additional Kconfig settings and a few optimizations would make it possible to significantly reduce this time. Cheers, -Kevin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 22/05/2015 13:12, Daniel P. Berrange wrote: In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. I think removing the ISA/LPC bridge is hard. It includes the real-time clock and fw_cfg, for example. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On Mon, May 25, 2015 at 02:52:51PM +0200, Paolo Bonzini wrote: On 23/05/2015 01:23, Kevin O'Connor wrote: I'm curious if you've tried profiling SeaBIOS to see where it is spending unnecessary time? No, I really wanted to get the absolute minimum time needed to get to the kernel. I announced it publicly because I think it's also instructive. I wonder if a stripped down SeaBIOS could obtain sufficient performance. It most likely could. However, some of the features (e.g. booting vmlinuz from parallel flash or, in the future, from persistent memory) probably do not fit SeaBIOS very well. One would have to redo them otherwise, for example using a DMA interface in fw_cfg. I think adding vmlinuz and/or multiboot support to SeaBIOS would be generally useful (and not difficult). If a kernel was placed in memory (or memory-mapped flash) then I think it would be reasonable to have SeaBIOS obtain its address from (an unoptimized) fw_cfg and deploy it. -Kevin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On Mon, May 25, 2015 at 09:21:49AM +0300, Vasiliy Tolstov wrote:
2015-05-23 6:55 GMT+03:00 Kevin O'Connor ke...@koconnor.net:
Out of curiosity, I ran some additional timing tests. With SeaBIOS
fully stripped down (via Kconfig), it takes ~20ms to get to the boot
phase on my old AMD system. Of that 20ms, ~7ms is to enable shadow
ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci
init, and ~6ms is to make the shadow ram area read-only. The time in
the remaining parts of the SeaBIOS code is so small that it's hard to
measure.
Can you share config for seabios? As i understand i can safety to
remove keybord, ps2, usb, ata/ahci and leave only virtio (in case of
using qemu to boo linux/freebsd/windows systems) ?
To get to 20ms, I basically disabled everything. (Actually, after
sending my last email I found I cound get to 16ms by setting
CONFIG_WRITABLE_UPPERMEMORY=y - it avoids much of the shadow ram
read-only cost.) It's not possible to actually boot anything with
this config - I was instead timing to the point where one could add a
multiboot or vmlinux boot capability to SeaBIOS.
To get to these times I disabled debug messages themselves (they take
a few ms of the boot time). So, to time events without debug messages
I added a few outb() calls to seabios at critical points. See patch
and config below.
-Kevin
--- a/src/post.c
+++ b/src/post.c
@@ -167,11 +167,15 @@ platform_hardware_setup(void)
// Init base pc hardware.
pic_setup();
mathcp_setup();
+outb('3', 0x402); outb('\n', 0x402);
timer_setup();
+outb('4', 0x402); outb('\n', 0x402);
clock_setup();
// Platform specific setup
+outb('5', 0x402); outb('\n', 0x402);
qemu_platform_setup();
+outb('6', 0x402); outb('\n', 0x402);
coreboot_platform_setup();
}
@@ -200,6 +204,7 @@ startBoot(void)
// Clear low-memory allocations (required by PMM spec).
memset((void*)BUILD_STACK_ADDR, 0, BUILD_EBDA_MINIMUM - BUILD_STACK_ADDR);
+outb('8', 0x402); outb('\n', 0x402);
dprintf(3, Jump to int19\n);
struct bregs br;
memset(br, 0, sizeof(br));
@@ -247,6 +252,7 @@ maininit(void)
prepareboot();
// Write protect bios memory.
+outb('7', 0x402); outb('\n', 0x402);
make_bios_readonly();
// Invoke int 19 to start boot process.
@@ -331,12 +337,14 @@ handle_post(void)
serial_debug_preinit();
debug_banner();
+outb('1', 0x402); outb('\n', 0x402);
// Check if we are running under Xen.
xen_preinit();
// Allow writes to modify bios area (0xf)
make_bios_writable();
+outb('2', 0x402); outb('\n', 0x402);
// Now that memory is read/writable - start post process.
dopost();
#
# Automatically generated file; DO NOT EDIT.
# SeaBIOS Configuration
#
#
# General Features
#
# CONFIG_COREBOOT is not set
CONFIG_QEMU=y
# CONFIG_CSM is not set
CONFIG_QEMU_HARDWARE=y
# CONFIG_XEN is not set
# CONFIG_THREADS is not set
# CONFIG_RELOCATE_INIT is not set
# CONFIG_BOOTMENU is not set
# CONFIG_BOOTORDER is not set
CONFIG_ENTRY_EXTRASTACK=y
CONFIG_MALLOC_UPPERMEMORY=y
CONFIG_ROM_SIZE=0
#
# Hardware support
#
# CONFIG_ATA is not set
# CONFIG_AHCI is not set
# CONFIG_SDCARD is not set
# CONFIG_VIRTIO_BLK is not set
# CONFIG_VIRTIO_SCSI is not set
# CONFIG_PVSCSI is not set
# CONFIG_ESP_SCSI is not set
# CONFIG_LSI_SCSI is not set
# CONFIG_MEGASAS is not set
# CONFIG_FLOPPY is not set
# CONFIG_PS2PORT is not set
# CONFIG_USB is not set
# CONFIG_SERIAL is not set
# CONFIG_LPT is not set
# CONFIG_USE_SMM is not set
CONFIG_MTRR_INIT=y
CONFIG_PMTIMER=y
#
# BIOS interfaces
#
CONFIG_DRIVES=y
CONFIG_CDROM_BOOT=y
CONFIG_CDROM_EMU=y
CONFIG_PCIBIOS=y
CONFIG_APMBIOS=y
CONFIG_PNPBIOS=y
# CONFIG_OPTIONROMS is not set
CONFIG_BOOT=y
CONFIG_KEYBOARD=y
CONFIG_KBD_CALL_INT15_4F=y
CONFIG_MOUSE=y
CONFIG_S3_RESUME=y
# CONFIG_VGAHOOKS is not set
# CONFIG_DISABLE_A20 is not set
CONFIG_WRITABLE_UPPERMEMORY=y
# CONFIG_TCGBIOS is not set
#
# BIOS Tables
#
# CONFIG_PIRTABLE is not set
# CONFIG_MPTABLE is not set
# CONFIG_SMBIOS is not set
# CONFIG_ACPI is not set
# CONFIG_FW_ROMFILE_LOAD is not set
#
# VGA ROM
#
CONFIG_NO_VGABIOS=y
# CONFIG_VGA_STANDARD_VGA is not set
# CONFIG_VGA_CIRRUS is not set
# CONFIG_VGA_BOCHS is not set
# CONFIG_VGA_GEODEGX2 is not set
# CONFIG_VGA_GEODELX is not set
# CONFIG_BUILD_VGABIOS is not set
CONFIG_VGA_FIXUP_ASM=y
CONFIG_VGA_EXTRA_STACK_SIZE=512
#
# Debugging
#
CONFIG_DEBUG_LEVEL=0
Re: Announcing qboot, a minimal x86 firmware for QEMU
On Fri, May 22, 2015 at 07:23:27PM -0400, Kevin O'Connor wrote: On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote: Some of you may have heard about the Clear Containers initiative from Intel, which couple KVM with various kernel tricks to create extremely lightweight virtual machines. The experimental Clear Containers setup requires only 18-20 MB to launch a virtual machine, and needs about 60 ms to boot. Now, as all of you probably know, QEMU is great for running Windows or legacy Linux guests, but that flexibility comes at a hefty price. Not only does all of the emulation consume memory, it also requires some form of low-level firmware in the guest as well. All of this adds quite a bit to virtual-machine startup times (500 to 700 milliseconds is not unusual). Right? In fact, it's for this reason that Clear Containers uses kvmtool instead of QEMU. No, wrong! In fact, reporting bad performance is pretty much the same as throwing down the gauntlet. Enter qboot, a minimal x86 firmware that runs on QEMU and, together with a slimmed-down QEMU configuration, boots a virtual machine in 40 milliseconds[2] on an Ivy Bridge Core i7 processor. Hi Paolo, I'm curious if you've tried profiling SeaBIOS to see where it is spending unnecessary time? I wonder if a stripped down SeaBIOS could obtain sufficient performance. The page at http://seabios.org/Debugging#Timing_debug_messages describes how to do basic profiling via timing of debug messages. The default SeaBIOS build takes ~180ms on my (old AMD) system. But, by removing drivers and options via Kconfig I was able to bring it down to ~25ms. I suspect some additional Kconfig settings and a few optimizations would make it possible to significantly reduce this time. Out of curiosity, I ran some additional timing tests. With SeaBIOS fully stripped down (via Kconfig), it takes ~20ms to get to the boot phase on my old AMD system. Of that 20ms, ~7ms is to enable shadow ram, 2ms is to calibrate the cpu timestamp counter, 4ms is for pci init, and ~6ms is to make the shadow ram area read-only. The time in the remaining parts of the SeaBIOS code is so small that it's hard to measure. The above delays could be removed without much work: It would be trivial to add kconfig options to support disabling the pci init and shadow read-only functionality. The cpu timestamp calibrations could be reworked so that the existence of the ACPI timer was checked first so as to avoid that work. The enabling of shadow ram could also be bypassed if QEMU was changed to start with a ram based shadow instead of a rom shadow (since QEMU doesn't fully implement the chipset shadow registers, it's really pointless to start in rom shadow mode anyway). -Kevin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote: Some of you may have heard about the Clear Containers initiative from Intel, which couple KVM with various kernel tricks to create extremely lightweight virtual machines. The experimental Clear Containers setup requires only 18-20 MB to launch a virtual machine, and needs about 60 ms to boot. Now, as all of you probably know, QEMU is great for running Windows or legacy Linux guests, but that flexibility comes at a hefty price. Not only does all of the emulation consume memory, it also requires some form of low-level firmware in the guest as well. All of this adds quite a bit to virtual-machine startup times (500 to 700 milliseconds is not unusual). Right? In fact, it's for this reason that Clear Containers uses kvmtool instead of QEMU. No, wrong! In fact, reporting bad performance is pretty much the same as throwing down the gauntlet. Enter qboot, a minimal x86 firmware that runs on QEMU and, together with a slimmed-down QEMU configuration, boots a virtual machine in 40 milliseconds[2] on an Ivy Bridge Core i7 processor. Hi Paolo, I'm curious if you've tried profiling SeaBIOS to see where it is spending unnecessary time? I wonder if a stripped down SeaBIOS could obtain sufficient performance. The page at http://seabios.org/Debugging#Timing_debug_messages describes how to do basic profiling via timing of debug messages. The default SeaBIOS build takes ~180ms on my (old AMD) system. But, by removing drivers and options via Kconfig I was able to bring it down to ~25ms. I suspect some additional Kconfig settings and a few optimizations would make it possible to significantly reduce this time. Cheers, -Kevin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote: Some of you may have heard about the Clear Containers initiative from Intel, which couple KVM with various kernel tricks to create extremely lightweight virtual machines. The experimental Clear Containers setup requires only 18-20 MB to launch a virtual machine, and needs about 60 ms to boot. Now, as all of you probably know, QEMU is great for running Windows or legacy Linux guests, but that flexibility comes at a hefty price. Not only does all of the emulation consume memory, it also requires some form of low-level firmware in the guest as well. All of this adds quite a bit to virtual-machine startup times (500 to 700 milliseconds is not unusual). Right? In fact, it's for this reason that Clear Containers uses kvmtool instead of QEMU. No, wrong! In fact, reporting bad performance is pretty much the same as throwing down the gauntlet. On the QEMU side of things I wonder if there is scope for taking AArch64's 'virt' machine type concept and duplicating it on all architectures. It would be nice to have a common minimal machine type on all architectures that discards all legacy platform stuff and focuses on the minimum needed to run modern virtual machine optimized guest OS. People would always know that a machine type called 'virt' was the minimal virtualization platform, while the others all target emulation of realworld (legacy) baremetal platforms. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 22 May 2015 at 12:01, Daniel P. Berrange berra...@redhat.com wrote: On the QEMU side of things I wonder if there is scope for taking AArch64's 'virt' machine type concept and duplicating it on all architectures. Experience suggests that holding the line on minimal is really quite tricky, though -- there's always one more thing that somebody really wants to add... -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On Fr, 2015-05-22 at 12:21 +0100, Peter Maydell wrote: On 22 May 2015 at 12:12, Daniel P. Berrange berra...@redhat.com wrote: Yep, it is hard saying no - but I'd think as long as it was possible to add the extra features using -device, it ought to be practical to keep a virt machine types -nodefaults -nodefconfig base setup pretty minimal. Mmm, but -device only works for pluggable devices really. We don't have a coherent mechanism for saying put the PS/2 keyboard controller into the system at its usual IO ports on the command line. Do we need that in the first place? You can plugin a usb keyboard today. You'll be able to plugin a virtio keyboard soon. I think alot of hardware where this applies to is the legacy stuff we want to get rid of for '-M virt' ... cheers, Gerd -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 22 May 2015 at 12:12, Daniel P. Berrange berra...@redhat.com wrote: Yep, it is hard saying no - but I'd think as long as it was possible to add the extra features using -device, it ought to be practical to keep a virt machine types -nodefaults -nodefconfig base setup pretty minimal. Mmm, but -device only works for pluggable devices really. We don't have a coherent mechanism for saying put the PS/2 keyboard controller into the system at its usual IO ports on the command line. -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On Fri, May 22, 2015 at 12:04:54PM +0100, Peter Maydell wrote: On 22 May 2015 at 12:01, Daniel P. Berrange berra...@redhat.com wrote: On the QEMU side of things I wonder if there is scope for taking AArch64's 'virt' machine type concept and duplicating it on all architectures. Experience suggests that holding the line on minimal is really quite tricky, though -- there's always one more thing that somebody really wants to add... Yep, it is hard saying no - but I'd think as long as it was possible to add the extra features using -device, it ought to be practical to keep a virt machine types -nodefaults -nodefconfig base setup pretty minimal. In particular I don't see why we need to have a SATA controller and ISA/LPC bridge in every virt machine - root PCI bus only should be possible, as you can provide disks via virtio-blk or virtio-scsi and serial, parallel, mouse, floppy via PCI devices and/or by adding a USB bus in the cases where you really need one. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On Fri, May 22, 2015 at 12:21:27PM +0100, Peter Maydell wrote: On 22 May 2015 at 12:12, Daniel P. Berrange berra...@redhat.com wrote: Yep, it is hard saying no - but I'd think as long as it was possible to add the extra features using -device, it ought to be practical to keep a virt machine types -nodefaults -nodefconfig base setup pretty minimal. Mmm, but -device only works for pluggable devices really. We don't have a coherent mechanism for saying put the PS/2 keyboard controller into the system at its usual IO ports on the command line. Oh, I didn't neccessarily mean that we'd need the ability to add a ps/2 keyboard via -device. I meant that there just need to be able to add /some/ kind of keyboard. eg we have a usb-kbd device that could potentially fill that role. Likewise for mouse pointer. Serial ports, etc. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
Peter Maydell peter.mayd...@linaro.org writes: On 22 May 2015 at 12:12, Daniel P. Berrange berra...@redhat.com wrote: Yep, it is hard saying no - but I'd think as long as it was possible to add the extra features using -device, it ought to be practical to keep a virt machine types -nodefaults -nodefconfig base setup pretty minimal. Mmm, but -device only works for pluggable devices really. We don't have a coherent mechanism for saying put the PS/2 keyboard controller into the system at its usual IO ports on the command line. ... yet. The qdev long-term vision has always been to support starting with an empty board, then add device models and their wiring. Unfortunately, progress towards that goal has been slow. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
Hi, qboot is available at git://github.com/bonzini/qboot.git. Firmware repo has packages now. https://www.kraxel.org/repos/firmware.repo https://www.kraxel.org/repos/jenkins/qboot/ enjoy, Gerd -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 05/21/2015 07:21 PM, Paolo Bonzini wrote: On 21/05/2015 17:48, Avi Kivity wrote: Lovely! Note you have memcpy.o instead of memcpy.c. Doh, and it's not used anyway. Check the repository, and let me know if OSv boots with it (it probably needs ACPI; Linux doesn't boot virtio without ACPI). Yes, it requires ACPI. We don't implement the pre-ACPI bootstrap methods. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Announcing qboot, a minimal x86 firmware for QEMU
Some of you may have heard about the Clear Containers initiative from Intel, which couple KVM with various kernel tricks to create extremely lightweight virtual machines. The experimental Clear Containers setup requires only 18-20 MB to launch a virtual machine, and needs about 60 ms to boot. Now, as all of you probably know, QEMU is great for running Windows or legacy Linux guests, but that flexibility comes at a hefty price. Not only does all of the emulation consume memory, it also requires some form of low-level firmware in the guest as well. All of this adds quite a bit to virtual-machine startup times (500 to 700 milliseconds is not unusual). Right? In fact, it's for this reason that Clear Containers uses kvmtool instead of QEMU. No, wrong! In fact, reporting bad performance is pretty much the same as throwing down the gauntlet. Enter qboot, a minimal x86 firmware that runs on QEMU and, together with a slimmed-down QEMU configuration, boots a virtual machine in 40 milliseconds[2] on an Ivy Bridge Core i7 processor. qboot is available at git://github.com/bonzini/qboot.git. In all the glory of its 8KB of code, it brings together various existing open source components: * a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS * a couple hardware initialization routines written mostly from scratch but with good help from SeaBIOS source code * a minimal 32-bit libc based on kvm-unit-tests * the Linux loader from QEMU itself The repository has more information on how to achieve fast boot times, and examples of using qboot. Right now there is a limit of 8 MB for vmlinuz+initrd+cmdline, which however should be enough for initrd-less containers. The first commit to qboot is more or less 24 hours old, so there is definitely more work to do, in particular to extract ACPI tables from QEMU and present them to the guest. This is probably another day of work or so, and it will enable multiprocessor guests with little or no impact on the boot times. SMBIOS information is also available from QEMU. On the QEMU side, there is no support yet for persistent memory and the NFIT tables from ACPI 6.0. Once that (and ACPI support) is added, qboot will automatically start using it. Happy hacking! Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On Thu, May 21, 2015 at 03:51:43PM +0200, Paolo Bonzini wrote: On the QEMU side, there is no support yet for persistent memory and the NFIT tables from ACPI 6.0. Once that (and ACPI support) is added, qboot will automatically start using it. We are working on adding NFIT support into virtual bios. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 05/21/2015 04:51 PM, Paolo Bonzini wrote: Some of you may have heard about the Clear Containers initiative from Intel, which couple KVM with various kernel tricks to create extremely lightweight virtual machines. The experimental Clear Containers setup requires only 18-20 MB to launch a virtual machine, and needs about 60 ms to boot. Now, as all of you probably know, QEMU is great for running Windows or legacy Linux guests, but that flexibility comes at a hefty price. Not only does all of the emulation consume memory, it also requires some form of low-level firmware in the guest as well. All of this adds quite a bit to virtual-machine startup times (500 to 700 milliseconds is not unusual). Right? In fact, it's for this reason that Clear Containers uses kvmtool instead of QEMU. No, wrong! In fact, reporting bad performance is pretty much the same as throwing down the gauntlet. Enter qboot, a minimal x86 firmware that runs on QEMU and, together with a slimmed-down QEMU configuration, boots a virtual machine in 40 milliseconds[2] on an Ivy Bridge Core i7 processor. qboot is available at git://github.com/bonzini/qboot.git. In all the glory of its 8KB of code, it brings together various existing open source components: * a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS * a couple hardware initialization routines written mostly from scratch but with good help from SeaBIOS source code * a minimal 32-bit libc based on kvm-unit-tests * the Linux loader from QEMU itself The repository has more information on how to achieve fast boot times, and examples of using qboot. Right now there is a limit of 8 MB for vmlinuz+initrd+cmdline, which however should be enough for initrd-less containers. The first commit to qboot is more or less 24 hours old, so there is definitely more work to do, in particular to extract ACPI tables from QEMU and present them to the guest. This is probably another day of work or so, and it will enable multiprocessor guests with little or no impact on the boot times. SMBIOS information is also available from QEMU. On the QEMU side, there is no support yet for persistent memory and the NFIT tables from ACPI 6.0. Once that (and ACPI support) is added, qboot will automatically start using it. Happy hacking! Lovely! Note you have memcpy.o instead of memcpy.c. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing qboot, a minimal x86 firmware for QEMU
On 2015-05-21 15:51, Paolo Bonzini wrote: Some of you may have heard about the Clear Containers initiative from Intel, which couple KVM with various kernel tricks to create extremely lightweight virtual machines. The experimental Clear Containers setup requires only 18-20 MB to launch a virtual machine, and needs about 60 ms to boot. Now, as all of you probably know, QEMU is great for running Windows or legacy Linux guests, but that flexibility comes at a hefty price. Not only does all of the emulation consume memory, it also requires some form of low-level firmware in the guest as well. All of this adds quite a bit to virtual-machine startup times (500 to 700 milliseconds is not unusual). Right? In fact, it's for this reason that Clear Containers uses kvmtool instead of QEMU. No, wrong! In fact, reporting bad performance is pretty much the same as throwing down the gauntlet. Enter qboot, a minimal x86 firmware that runs on QEMU and, together with a slimmed-down QEMU configuration, boots a virtual machine in 40 milliseconds[2] on an Ivy Bridge Core i7 processor. qboot is available at git://github.com/bonzini/qboot.git. In all the glory of its 8KB of code, it brings together various existing open source components: * a minimal (really minimal) 16-bit BIOS runtime based on kvmtool's own BIOS * a couple hardware initialization routines written mostly from scratch but with good help from SeaBIOS source code * a minimal 32-bit libc based on kvm-unit-tests * the Linux loader from QEMU itself The repository has more information on how to achieve fast boot times, and examples of using qboot. Right now there is a limit of 8 MB for vmlinuz+initrd+cmdline, which however should be enough for initrd-less containers. The first commit to qboot is more or less 24 hours old, so there is definitely more work to do, in particular to extract ACPI tables from QEMU and present them to the guest. This is probably another day of work or so, and it will enable multiprocessor guests with little or no impact on the boot times. SMBIOS information is also available from QEMU. On the QEMU side, there is no support yet for persistent memory and the NFIT tables from ACPI 6.0. Once that (and ACPI support) is added, qboot will automatically start using it. Happy hacking! Incidentally, I did something similar these days to get Linux booting in Jailhouse non-root cells, i.e without BIOS and almost no hardware except memory, cpus and pci devices. Yes, requires a bit pv for Linux, but really little. Not aiming for speed (yet), just for less hypervisor work. Maybe there are some milliseconds to save when cutting off more hardware in an analogous way... PV pat^Whacks are here: http://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse. The boot loader is a combination of a python script [1] (result can be saved and reused - replaces ACPI) and really few lines of code [2][3]. Jan [1] https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/tools/jailhouse-cell-linux [2] https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/inmates/lib/x86/header.S [3] https://github.com/siemens/jailhouse/blob/wip/linux-x86-inmate/inmates/tools/x86/linux-loader.c -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Announcing qboot, a minimal x86 firmware for QEMU
On 21/05/2015 17:48, Avi Kivity wrote: Lovely! Note you have memcpy.o instead of memcpy.c. Doh, and it's not used anyway. Check the repository, and let me know if OSv boots with it (it probably needs ACPI; Linux doesn't boot virtio without ACPI). Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
