Jason Venner wrote:
I have found that HOD writes a series of log files to directories on
the virtual cluster master, if you specify log directories.
The interesting part is figuring out which machine was the virtual
cluster master, if you have a decent sized pool of machines.
Can you explain what you mean by 'virtual cluster master' ? I guess you
could mean the node which ran the 'ringmaster' process - the master
process in a hod virtual cluster, or the hadoop 'jobtracker'. Both this
information is stored by hod for an allocated cluster. You could
retrieve it by using
hod -o list,
and
hod -o "info cluster-dir"
If the cluster is deallocated, you can get the ringmaster node still by
using torque commands:
qstat (to get torque job ids)
and
qstat -n <jobid>
This will print a list of nodes, the first one is the ringmaster.
It would be so nice if HOD could copy this back to the submission node.
I suppose we could configure syslog up but then we have to configure
the syslogd on each of the submit hosts to accept from any compute
node...
Syslog configuration has a bug which will be addressed in Hadoop 0.16.1.
Thanks
hemanth
