Rather than spending megabucks on VMWare ESX with VMotion, we wrote a layer on top of Xen that does live migration, failover, monitoring, etc. It is open sourced as Ganeti: http://code.google.com/p/ganeti/ Ganeti 2.0 has a plug-in architecture so you can choose VM system (Xen vs. KVM), storage (disk vs. SAN vs. DRBD), and so on.
It is a very nice system for building your own cloud. For physical machines we're considering adding a new plug-in to run UML (usermode linux) or Containers so people that need the full machine are still under Ganeti. This would be a simple matter of setting up a new template. For purely physical machines, we do a couple tricks. We want to be able to wipe and reload a machine with an API, just like we can with Ganeti (or EC2 for that matter). So here's what we do. First, the hardware is configured to first try to PXE boot, if that fails they boot on their local disk. Machines that are "up" are not listed in DHCP so the PXE boot fails and we boot off the local disk. If we need to wipe and reload a machine, the software (not Ganeti, an internal tool) will temporarily add a DHCP entry for that machine, and wait for it to reboot. It will then PXE boot, find our image installer, which when it is done tells the DHCP system to withdraw the entry for that machine so it boots off local disk again. This way we can progamatically wipe and reload a physical machine about as easily as using the EC2 API to allocate and start a new virtual machine. It is just slower and, obviously, we haven't cloned the EC2 API. You may be wondering how we get the machine to reboot. First we try "ssh HOST reboot", if that fails we ask the user to reboot the machine. We also do something (not sure what) so that when the kernel dies the machine reboots rather than just freeze up. With Ganeti you can set up your own virtual machine cloud in an afternoon. The tricks we use for physical machines are fairly common at big data centers but I don't know if there are any open source implementations. Tom _______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
