Edward Ned Harvey <[email protected]> writes: > I can't believe, after all these years, and countless deployments, I'm still > so dramatically dissatisfied with virtualization. Wondering if anyone knows > something that doesn't suck. I may have some incomplete or incorrect > information, so please - comment your heart out. ;-)
I'm running north of 700 guests on something like 15 8 core 32GiB ram servers; some of them have uptimes around 2 years. this is an environment where I'll give anyone with a paypal account a VM with a console and rebooter; some of my guests are actively attempting to abuse the system; none of them are cooperating with me. The system works fairly well. Sure, I have to watch for abuse, but it's the same way if I was hosting dedicated servers. I give each guest 1 vcpu, and I reserve a physical cpu for the dom0, and I don't have to worry about performance. Sure, I have a few jokers running mprime, but really the Xen scheduler is pretty good, and it keeps them contained. Anyhow, here are my thoughts on xen and virtualization: First, in my experience, all 'full virtualization' products suck. They are slow and buggy. I only support paravirtualized kernels. That said, I will support any paravirtualized kernel; I give the user access to a serial console and PVGRUB. The users can run a (possibly hostile) kernel. This doesn't bother me. Everything I care about has paravirtualized guest kernels, so that's okay. If you are using windows, nothing I say applies to you. Also, it sounds like you want a more 'desktop-like' system than I do. I use the open-source xen hypervisor (the one included in RHEL 5.4 is kinda okay, but I currently use the Xen 3.4 hypervisor from xen.org.) The commercial xenserver product might be worth a shot if you really need graphical tools. It's free, I hear, for the basic version. But the management console is windows only. But, in my opinion, graphical tools are overrated. xm is really the only management tool you need. ... > 8. Virtual Machine Manager and Xen Personally, I think the RHEL tools for managing virtualization suck. I use the xm tools directly. I avoid virt-manager and the whole stack, especially since RedHat has become so Xen-hostile, before KVM is ready for anything but 'let's spin up a test system real quick, oh, let's do something slightly faster than Kqemu' > a. I'm not sure if the problem is more VMM or Xen. They both suck. > Maybe some of this will improve with KVM, but you're still stuck with VMM KVM has a long way to go. first, it's only full virtualization, so personally I believe it will likely always suck. Even then, it has a long way to come before it is as stable and performant as the (unusably slow and buggy, for my purpouses) xen HVM mode. > b. Once a CD is inserted (let's say an ISO file is used) you can't > change CD's while the machine is on. You have to shutdown the machine in > order to change the ISO file. This is particularly a problem if you install > the OS from a set of ISO CD images. yeah. I do a 'network install' from within the DomU. I don't drive CDs down to the co-lo, either. > c. There's some serious memory leakage going on here. Sometimes one > of my virtual guest machine servers starts behaving strange, and users ask > me to check it. So I open the console of Guest A, and get the screen of > Guest B. So I try to open the console of Guest B, only to discover Guest B > is now dead. Solution: Reboot. The only time I see this in paravirtualized xen is when someone is trying to do something crazy in a small ram domain without swap. Add swap or ram and it works. (personally, I am irritated by Linux's massively aggressive memory overcommit) speaking of memory overcommit, all the ballooning shit in xen? It's crap. don't use it. set dom0-min-mem in xend-config.sxp to the same value as dom0_mem on the xen kernel command line in the grub config. Very important. this number should be somewhere between 512M and 1024M. Make sure to give the Dom0 swap, and to avoid running things in the Dom0. > d. Xen seems to crash my guests . Almost monthly. I'm going to guess > the average is once in 5-6 weeks. Makes me pee my bed at night. If you are not using HVM Mode, I would strongly suspect bad hardware. As far as I can tell, paravirtualized mode Xen is pretty solid (except for the aformentioned problem of running a VM with little ram and not enough swap, which is mostly a problem of having low prices and cheap customers. Before I got rid of it, I was getting a lot of 64MiB customers, even though for a dollar more, I'd give them 128Mib ram.) > e. Again - I'm not sure if this is meant to be just a desktop product > or not - but you can only get a guest console from the local host machine. Don't use the graphical console. However, if you must, you can get vnc access to the virtual framebuffer: grep vnc /etc/xen/xmexample.hvm will give you the bits required to connect to vnc remotely. For obvious security reasons, it defaults to only listening on 127.0.0.1 (locally, you can also have a SDL console, which is obviously better.) Personally, I always use xm console domain which gives you access to the xen virtual serial console. Takes some work to setup in hvm mode, but works there, too. In paravirt mode, it's dead easy. -- Luke S. Crawford http://nostarch.com/xen.htm http://prgmr.com/xen/ _______________________________________________ Tech mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
