Jason Venner wrote:
I have found that HOD writes a series of log files to directories on the virtual cluster master, if you specify log directories. The interesting part is figuring out which machine was the virtual cluster master, if you have a decent sized pool of machines.
In torque set up, HOD starts the "virtual cluster master" called ringmaster on the first node in the node list returned by torque. So doing a "qstat -f torque-jobid" or better "qstat -an jobid" will show you the list of nodes allocated for the current hod allocation, the first of them being the ringmaster.
It would be so nice if HOD could copy this back to the submission node.
HOD currently doesn't copy any logs, neither of the master nor of the hodrings, to the submission node. Ideally, one should be able to figure out anything wrong just by looking at the error codes.
I suppose we could configure syslog up but then we have to configure the syslogd on each of the submit hosts to accept from any compute node...
Yeah, that should be the case when using syslogd. But a better way would be to set up syslog-ng which would need setup at one centralized location, and let hod write all logs to that. JFYI, there is one issue still to be resolved ( HADOOP-2809 <https://issues.apache.org/jira/browse/HADOOP-2809> ); once it is done you should be able to log to syslog/sylog-ng.
Hope that helps, Thanks, Vinod
