Alex Zheng wrote:
I am a newer for Hadoop, and am reading the code of Hadoop for a week
now i am very puzzled by the relation of so many classes after i run :
bin/start-all.sh
I know there are JobTrackerInstrumentation, JobTracker, Namenode etc so what
is the order of their initialization?
and after bin/start-all.sh and before i run any job, what exits in the
system?
run jps -v to see what's up and about, netstat -p to list ports in use
by the different processes.
The nodes are all designed to spin a bit waiting for their dependencies
to come up; you don't need to bring them up in a strict order (which
would be namenode-datanode(s)-jobtracker-tasktracker(s)) for a full MR
cluster.
I have tests that poll for the various ports to be open before
submitting work, and they sometimes get unhappy if you try submitting
jobs straight after the job tracker appears live. If you are going to
spin waiting for a job tracker to be visible, I would sleep a few
seconds after it's IPC port opens up before sending in work. This is
clearly some race condition, but not anything I've sat down to look at,
as it's only a startup and a 10s sleep makes it go away
