Integration of Virtualization (such as Xen) with Hadoop tools
-------------------------------------------------------------
Key: HADOOP-6961
URL: https://issues.apache.org/jira/browse/HADOOP-6961
Project: Hadoop Common
Issue Type: New Feature
Environment: All
Reporter: Saikat Kanjilal
How does one integrate sandboxing of arbitrary user code in C++ and other
languages in a VM such as Xen with the Hadoop framework? How does this interact
with SGE, Torque, Condor?
As each individual machine has more and more cores/cpus, it makes sense to
partition each machine into multiple virtual machines. That gives us a number
of benefits:
By assigning a virtual machine to a datanode, we effectively isolate the
datanode from the load on the machine caused by other processes, making the
datanode more responsive/reliable.
With multiple virtual machines on each machine, we can lower the granularity of
hod scheduling units, making it possible to schedule multiple tasktrackers on
the same machine, improving the overall utilization of the whole clusters.
With virtualization, we can easily snapshot a virtual cluster before releasing
it, making it possible to re-activate the same cluster in the future and start
to work from the snapshot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
