Hey Alvin,


First, I am wondering how the allocation actually works. For example, it seems like the hod interface allows me to specify which nodes to run on (in cluster_dir/hadoop-site.xml", and how many nodes to run on (-o option in hod). When I execute this command, do I have to know in advance that all the nodes in my cluster_dir/hadoop-site.xml are available for HOD to use (i.e., HOD might feel free to wipe anything on it)? And if the number of requested nodes have is smaller than the hosts specified in hadoop-site.xml, how does HOD decide which nodes to run on?


On a first note, can you please tell which documentation you are looking at. 'Coz hod interface is cleaned up and -o option is part of old interface. You can find the latest HOD documentation at http://hadoop.apache.org/core/docs/r0.16.3/hod.html

Hod interface lets you tell how many nodes nodes you want, and not the actual nodes. HOD obtains the list of nodes and thus gets its components (hodring and ringmaster) running by talking to a resource manager, torque (http://www.clusterresources.com/pages/products/torque-resource-manager.php) being the only one that hod works with currently.

Also, HOD is advertised as: "a very useful tool for Hadoop developers and testers who need to share a physical cluster for testing their own Hadoop versions." Do these users need to actively coordinate which nodes they use for their virtual cluster or will HOD coordinate the different virtual clusters (i.e., each HOD virtual cluster is essentially a batch job on the physical cluster, and uses Torque/Maui to schedule)?


Your last point is right, each user can have his own HOD virtual cluster(s), with each cluster tied to a cluster directory that (s)he owns.

Please look here for a sample HOD session (http://hadoop.apache.org/core/docs/r0.16.3/hod_user_guide.html#A+typical+HOD+session).

HTH

-vinod


Reply via email to