Discover the Linux Kernel Virtual Machine

Learn the KVM architecture and advantages

M. Tim Jones (m...@mtjones.com), Consultant Engineer, Emulex

Summary: Linux® and flexibility go hand in hand, and the options for virtualization are no different. But recently, a change in the Linux virtualization landscape has appeared with the introduction of the Kernel virtual Machine, or KVM. KVM is the first virtualization solution to be part of the mainline Linux kernel (V2.6.20). KVM supports the virtualization of Linux guest operating systems -- even Windows® with hardware that is virtualization-aware. Learn about the architecture of the Linux KVM as well as why its tight integration with the kernel may change the way you use Linux.

Tag this!

Update My dW interests (Log in | What's this?) Skip to help for Update My dW interests

Date: 18 Apr 2007
Level: Intermediate
Also available in: Chinese Japanese Portuguese

Activity: 34518 views
Comments: 0 (Add comments)

Introduction

Virtualization is a concept that has been around for quite some time. Succinctly, it's the process of taking something and making it look like something else. Applying this concept to a computer system allows different users to view that single system differently (for example, a single computer that runs both Linux and Microsoft® Windows® concurrently). This is commonly called full virtualization.

KVM and kvm

In this article, I refer to KVM when talking about the Kernel Virtual Machine and kvm when discussing the hypervisor utility (used to kick off a new virtual machine).

Virtualization can also take a more complicated form, where a single computer appears as multiple architectures (to one user, it's a standard x86 platform; to another, it's an IBM Power PC® platform). This form of virtualization is commonly known as hardware emulation.

Finally, a simpler form of virtualization is operating system virtualization, in which a single computer runs numerous operating systems of the same type. This type of virtualization simply isolates several servers on a single operating system (which means that all must use the same type and version of the operating system). For more information on virtualization methods, see Resources.

Virtualization and para-virtualization

Two of the most common approaches to virtualization are full virtualization and para-virtualization. With full virtualization, a layer exists between the virtualized operating systems and the hardware as a way to arbitrate access. This layer is called a hypervisor, or virtual machine monitor (VMM). Para-virtualization is similar, but the hypervisor operates in a more cooperative fashion. This is because each guest operating system is aware that it's being virtualized, so each cooperates with the hypervisor to virtualize the underlying hardware.

Examples of full virtualization include the commercial virtualization solution VMware as well as the commercial IBM System z9 Virtual Machine (z/VM) operating system for IBM zSeries® computers. Para-virtualization is provided by Xen and User-Mode-Linux (UML). KVM is also considered a full virtualization solution, but I explore this later.

How virtualization works

Let's begin with a quick discussion of virtualization and the elements involved. At the bottom of a virtualization solution is the machine to be virtualized. This machine may or may not support virtualization directly, which then requires support by the next layer, called the hypervisor. The hypervisor, or VMM, serves as an abstraction between the platform hardware and the operating systems. In some cases, the hypervisor is an operating system; in this case, it's called the host operating system, as shown in Figure 1.

Figure 1. The layered abstraction of a virtualization

Above the hypervisor are the guest operating systems, also called virtual machines (VMs). These VMs are the isolated operating systems that view the underlying hardware platform as belonging to them. But in reality, the hypervisor provides them with this illusion.

Processor support for virtualization

The advantages of platform virtualization are so real that processor vendors have modified their silicon to support the method directly. Doing so allows the processor to directly support a hypervisor differently than that of a guest operating system. In addition to processor state (registers and so on) being managed differently for a VMM and a VM, the processor supports virtualization of I/O and interrupts. For more information, see Resources.

A problem with virtualization solutions today is that not all hardware supports virtualization cleanly. Older x86 processors produce different results for certain instructions depending upon the domain of execution. This creates a problem, because the hypervisor should execute in only the most protected domain. For this reason, virtualization solutions such as VMware will pre-scan code that is to execute to replace these instructions with trap instructions so that the hypervisor can handle them appropriately. Xen, supporting a cooperative method of virtualization, requires no modification, because the guest is aware that it's being virtualized and is modified. KVM simply ignores this problem and mandates that if you want virtualization, you'll have to run on newer hardware.

This probably seems a little heavy-handed at first, but considering that newer machines coming out today support virtualization (such as the Intel® VT and AMD SVM), it won't be long before this is the norm rather than the exception. For more information on the processors that support virtualization, see Resources and the sidebar, Processor support for virtualization.

The KVM hypervisor

Considering the time line of virtualization techniques, KVM is a relative newcomer. Several incumbent open source methods exist today, such as Xen, Bochs, UML, Linux-VServer, and coLinux, but KVM is receiving a surprising amount of exposure in the press. Further, KVM isn't really a full virtualization scheme by itself but rather a piece of a larger solution.

The approach that KVM takes is to turn a Linux kernel into a hypervisor simply by loading a kernel module. The kernel module exports a device called /dev/kvm, which enables a guest mode of the kernel (in addition to the traditional kernel and user modes). With /dev/kvm, a VM has its own address space separate from that of the kernel or any other VM that's running. Devices in the device tree (/dev) are common to all user-space processes. But /dev/kvm is different in that each process that opens it sees a different map (to support isolation of the VMs).

KVM source in the Linux kernel

You can find the source for the KVM in ./linux/drivers/kvm (in V2.6.20 and later). This directory contains the source files for the KVM as well as processor support files for the Intel and AMD extensions.

KVM then simply turns the Linux kernel into a hypervisor (when you install the kvm kernel module). Because the standard Linux kernel is the hypervisor, it benefits from the changes to the standard kernel (memory support, scheduler, and so on). Optimizations to these Linux components (such as the new O(1) scheduler in the 2.6 kernel) benefit both the hypervisor (the host operating system) and the Linux guest operating systems. But KVM isn't the first to do this. UML has been transforming the Linux kernel into a hypervisor for quite some time. With the kernel acting as a hypervisor, you can then start other operating systems, such as another Linux kernel or Windows.

KVM

With KVM installed, you can start guest operating systems in user-space. Each guest operating system is a single process of the host operating system (or hypervisor). Figure 2 provides a view of virtualization with KVM. At the bottom is a hardware platform that is virtualization capable (currently, this means an Intel VT or AMD-SVM processor). Running on the bare hardware is the hypervisor (the Linux kernel with the KVM module). This hypervisor looks just like a regular Linux kernel on which you can run any other application. But this kernel can also support guest operating systems, loaded through the kvm utility. Finally, a guest operating system can support the same applications that the host operating system supports.

Figure 2. The virtualization components with KVM

Recall that KVM is part of a virtualization solution. The processor provides the virtualization support directly (the ability to virtualize the processor for multiple operating systems). Memory is virtualized through kvm (which I discuss in the next section). Finally, I/O is virtualized through a lightly modified QEMU process (a copy of which executes with each guest operating system process).

KVM introduced a new process mode to Linux to the existing kernel and user modes. The new mode is called guest, and as the name suggests, it's used for execution of guest operating system code (or at least some of it). Recall that kernel mode represents the privileged mode for code execution, while user mode represents the non-privileged mode (for programs running outside the kernel). The modes of execution are then defined for different purposes based on what's running and for what purpose. Guest mode exists to execute guest operating system code but only for code that's non-I/O. Within guest mode are the standard two modes, so that a guest operating system runs in guest mode but supports the standard kernel and user modes for its kernel and user-space applications. The user mode of a guest operating system exists to perform I/O, which is independently managed.

Performing I/O from a guest operating system is provided with QEMU. QEMU is a platform virtualization solution that allows virtualization of an entire PC environment (including disks, graphic adapters, and network devices). Any I/O requests a guest operating system makes are intercepted and routed to the user mode to be emulated by the QEMU process.

KVM provides virtualization of memory through the /dev/kvm device. Each guest operating system has its own address space that is mapped when the guest is instantiated. The physical memory that is mapped for the guest operating system is actually virtual memory mapped into the process. A set of shadow page tables is maintained to support the translation from guest physical addresses to host physical addresses. The processor also supports the memory translation process by tapping into the hypervisor (host kernel) when an unmapped memory location is accessed.

Instantiating a new guest

Instantiation of a new guest operating system is provided by a utility called kvm. This utility works with the kvm module, using /dev/kvm to load a guest, associate it with the virtual disk (a regular file in the host operating system), and then boot it.

Control is supported through a set of ioctls that are performed on the /dev/kvm device. When the special file is first opened, a new VM object is created that is associated with a virtual CPU. You can then use several ioctls to create a virtual CPU, check kvm versions, create a memory region, and then start a virtual CPU. You do so by using the command kvm. I'll look at the kvm command and show examples of some of the supported ioctls in the following sections.

Using KVM

Using KVM is actually quite simple if your hardware supports it. You need a processor that has virtualization support: You can tell whether your system supports virtualization by looking at /proc/cpuinfo. This file specifies whether the vmx (Intel) or svm (AMD) extensions are supported.

Next, you need a Linux kernel with KVM support enabled. You do this in the kernel configuration under Device Drivers > Virtualization. You must also enable processor support for your environment. You must have the kvm and qemu user-space applications, as well. See Resources for more information.

Using a booted kernel with virtualization support enabled, the next step is to create a disk image for the guest operating system. You do so with qeumu-img, as shown below. Note that the size of the image is 4 GB, but using QEMU's copy-on-write format (qcow), the file will grow as needed instead of occupying the full 4 GB.

$ qemu-img create -f qcow vm-disk.img 4G

With your virtual disk created, load the guest operating system into it. The following example assumes that the guest operating system is on a CD-ROM. In addition to populating the virtual disk with the CD-ROM ISO image, you must boot the image when it's done.

$ kvm -no-acpi -m 384 -cdrom guestos.iso -hda vm-disk.img -boot d

Ari Kivity has written a set of test tools to test the KVM without needing the full device model. The following code snippet (from kvm-12/user/main.c) provides a high-level peek into starting a VM (see Listing 1). The control features are provided by ioctls in the kernel (specifically, in the file ./linux-2.6.20/drivers/kvm/kvm_main.c).

The call to kvm_init opens the /dev/kvm device, checks the version numbers (exported by the KVM kernel module), then allocates a KVM context object and fills in some callback functions. The kvm_create function sets and maps two memory regions, and then creates a virtual CPU (VCPU) with an ioctl (KVM_CREATE_VCPU).

The load_file function then loads the image into the address space for the given VM, which is then executed with a call to kvm_run (using ioctl KVM_RUN). While simple, this process illustrates how you can instantiate new guest operating systems with KVM.

Listing 1. Snippet of an application to test the KVM hypervisor

                
int main()
{
	void *vm_mem;


	kvm = kvm_init(&test_callbacks, 0);
	if (!kvm) {
	    fprintf(stderr, "kvm_init failed\n");
	    return 1;
	}
	if (kvm_create(kvm, 128 * 1024 * 1024, &vm_mem) < 0) {
	    kvm_finalize(kvm);
	    fprintf(stderr, "kvm_create failed\n");
	    return 1;
	}
	if (ac > 1)
	    if (strcmp(av[1], "-32") != 0)
		load_file(vm_mem + 0xf0000, av[1]);
	    else
		enter_32(kvm);
	if (ac > 2)
	    load_file(vm_mem + 0x100000, av[2]);
	kvm_show_regs(kvm, 0);

	kvm_run(kvm, 0);

	return 0;
}

Final thoughts

KVM is an interesting solution to the virtualization problem, but as it's the first one into the kernel, it's hard not to imagine that it will gain ground quickly for server virtualization. Other methods have been vying to get into the kernel for quite some time (such as UML and Xen), but because KVM required so few changes and was able to transform a standard kernel into a hypervisor, it's pretty clear why it was chosen.

Another advantage to KVM is that because it's part of the kernel itself, it can take advantage of kernel optimizations and advancements. This tends to future-proof the approach over other, independent hypervisor solutions. The two biggest drawbacks to KVM are that it requires newer virtualization-capable processors and a user-space QEMU process to provide I/O virtualization. But good or bad, KVM is in the kernel, which gives it a huge leap forward against the existing solutions.

Resources

Learn

There are numerous ways to provide virtualization, and you can learn more from the article, "Virtual Linux" (developerWorks, December 2006).
Several companies, including IBM, Intel, and AMD, provide virtualization hardware. You can check out their respective sites for more information.
For the latest news on KVM, check out the KVM wiki that's provided by Qumranet.
Even though KVM has been out for only a short time, you can find several articles exploring its performance in several areas. Two interesting articles are provided by Phoronix and KernelNewbies.
Another interesting article at KernelNewbies provides a great comparison of virtualization methods (including KVM).
The V2.6 Linux scheduler was created by Ingo Molnar to achieve O(1) scheduling. You can read more about this scheduler in "Inside the Linux scheduler" (M. Tim Jones, developerWorks, June 2006).
A cooperative virtualization method, coLinux allows you to run Linux on Windows.
QEMU is an open source processor emulator that also provides the PC environment virtualization capabilities for KVM.
In the developerWorks Linux zone, find more resources for Linux developers.
Stay current with developerWorks technical events and Webcasts.

Get products and technologies

To download the necessary applications for KVM, check out Debian.
Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.

Discuss

Check out developerWorks blogs and get involved in the developerWorks community.

About the author

M. Tim Jones is an embedded software architect and the author of GNU/Linux Application Programming, AI Application Programming, and BSD Sockets Programming from a Multilanguage Perspective. His engineering background ranges from the development of kernels for geosynchronous spacecraft to embedded systems architecture and networking protocols development. Tim is a Consultant Engineer for Emulex Corp. in Longmont, Colorado.

--
Subscription settings: http://groups.google.com/group/linuxkernelnewbies/subscribe?hl=en

[lk] Discover the Linux Kernel Virtual Machine

Discover the Linux Kernel Virtual Machine

KVM and kvm

Processor support for virtualization

KVM source in the Linux kernel

Reply via email to