I'm working on the Hadoop setup for a Mesos cluster. Here's what I take from
the documentation -- please correct if I'm wrong:
1. Each node (master and slaves) needs the following (replicated)
configuration files:
Mapred-site.xml (plugging in the URL of the Mesos master)
Hadoop-env.sh (with JAVA_HOME, PROTOBUF_JAR, MESOS_JAR,
MESOS_NATIVE_LIBRARY and HADOOP_CLASSPATH set)
Core-site.sh
And a "master" file and a "slaves" file (with the host names or IP
addresses of the master and slaves)
2. On the master, I should start a jobtracker when the cluster comes up.
3. Ports 9000 and 9001 need to be open on all instances.
I'm running the same AMI on all instances, so at least I know that the various
software components are all installed in the same place on all the machines in
the cluster.
Am I missing anything?
Thanks,
Jim Donahue
Adobe Systems
