----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4717/ -----------------------------------------------------------
Review request for mesos, Benjamin Hindman and Matei Zaharia. Summary ------- Earlier in the year I implemented a virtual machine isolation module. This module uses lib-virt to launch and manage virtual machine containers. The code is still rough and have done basic testing with the Spark example. This code works with the KVM (http://www.linux-kvm.org/page/Main_Page) virtual machine manager. I've placed the relevant code in a branch called mesos-vm, for now located at https://github.com/charlescearl/VirtualMesos. The code is based upon the mesos lxc isolation module that is located in src/slave/lxc_isolation_module.cpp/.hpp. My code based on the mesos master branch dated Wed Nov 23 12:02:07 2011 -0800, commit 059aabb2ec5bd7b20ed08ab9c439531a352ba3ec. I've included a patch for the relevant code included for the review. Suggestions appreciated on whether this is the appropriate branch/commit to patch against. Most of the implementation is contained in vm_isolation_module.cpp and vm_isolation_module.hpp and there are some minor additions in launcher to handle setup of the environment for the virtual machine. I use the libvirt (http://libvirt.org/) library, to manage the virtual machine container in which the jobs are executed. Dependencies The code has been tested on Ubuntu 11.04 and 11.10 and depends on libpython2.6 and libvirt0 Configuration of the virtual machine container The virtual machine invocation depends upon a few configuration assumptions: 1. ssh public keys installed on the container. I assume that the container is setup to allow password-less secure access. 2. Directory structure on the container matches the servant machine. For example, in invoking a spark executor, assume that the paths match the setup on the container host. Running it In the $MESOS_HOME/conf/mesos.conf file add the line isolation=vm to use the virtual machine isolation. The Mesos slave is invoked with the isolation parameter set to vm. For example sudo bin/mesos-slave -m mesos://master@mesos-host:5050 -w 9839 --isolation=vm Rough description of how it works The `vm_isolation_module` class forks a process that in turn launches a virtual machine. A routine located in bin called find_addr.pl is responsible for figuring out the IP address of the launched virtual machine. This is probably not portable since it is explicitly looking for entry in the virbr0 network. A script vmLauncherTemplate.sh located in bin assists the the vmLauncher method to setup the environment for launching tasks inside of the virtual machine. The vmLauncher method uses vmLauncherTemplate.sh to create a tasks specific shell vmLauncherTemplate-<task_id>.sh, which is copied to the running guest and used to run the executor inside the VM. This communicates with the slave on the host. Comments and suggestions on improvements and next directions are appreciated! Diffs ----- bin/find_addr.pl PRE-CREATION bin/killtree.sh PRE-CREATION bin/vmLauncher.sh PRE-CREATION bin/vmLauncherTemplate.sh PRE-CREATION src/config/config.hpp PRE-CREATION src/launcher/launcher.hpp b99b6d2 src/launcher/launcher.cpp 4422224 src/launcher/vm_mesos_launcher.cpp PRE-CREATION src/slave/isolation_module.cpp 5b7b4a2 src/slave/isolation_module_factory.cpp 6498945 src/slave/lxc_isolation_module.cpp ab0843a src/slave/main.cpp 9519ed2 src/slave/slave.cpp 21fc9f2 src/slave/vm_isolation_module.hpp PRE-CREATION src/slave/vm_isolation_module.cpp PRE-CREATION Diff: https://reviews.apache.org/r/4717/diff Testing ------- This was run with the spark example on single KVM virtual machine. Not tested extensively. Thanks, Charles
