On Mar 11, 2010, at 11:09 PM, Vladimir Mencl wrote: > On 03/11/2010 01:33 AM, Joseph Bester wrote: >>>> pbs_log_path=/usr/spool/PBS/server_logs >>>> >>>> Now, I could run the event-generator without any error. >> Do events for jobs started outside of globus show up in the SEG log file >> when you run the SEG? > > Hi Joe, > > Thanks for the reply. > > I've now had a fresh read through the documentation and I've realized > I'd skipped one crucial step in the SEG setup: > > cd $GLOBUS_LOCATION/setup/globus; ./setup-seg-job-manager.pl > > This is where I was breaking the setup: I thought I had hit a bug in the > setup scripts when globus-job-manager-event-generator was failing with > "Error: pbs not configured" and I had manually edited > globus-job-manager-seg.conf and added > "pbs_log_path=/usr/spool/PBS/server_logs" - while the correct line, > added by the setup-seg-job-manager.pl script, is > pbs_log_path=/opt/globus/var/globus-job-manager-seg-pbs > > Now.... it still doesn't work. I don't get any events at all. At least > now, when I try running a job with SEG enabled, the job manager does not > go on a memory hungry rampage. I guess what I did wrong was to tell > jobmanager to look for LRM-independent logs in the PBS server_logs > directory - which is what was turning it so mad. Uh oh.
Weird. I'll have to try that and see where it goes wrong. > But, I'm still stuck. The event-generator is not processing the PBS > logs at all. I run it with: > > $GLOBUS_LOCATION/sbin/globus-job-manager-event-generator -scheduler pbs > -pidfile /opt/globus/var/job-manager-seg-pbs.pid > > and even with strace and lsof, I can't see it doing any activity at all > - and it's not opening the PBS logs at all. I can see it does run the C > binary in the background: > /opt/globus/libexec/globus-scheduler-event-generator -s pbs -t 1 > > But even when I run the event generator with: > > export SEG_PBS_DEBUG=255 GLOBUS_ERROR_VERBOSE=1 GLOBUS_ERROR_OUTPUT=1 > $GLOBUS_LOCATION/sbin/globus-job-manager-event-generator -scheduler pbs > -pidfile /opt/globus/var/job-manager-seg-pbs.pid > > I only get: > > [INFO] Enter globus_l_pbs_increase_buffer > [INFO] Exit globus_l_pbs_increase_buffer > > and the output stops there. Any idea what's wrong? That seems an odd place for the output to stop. Any chance the globus-scheduler-event-generator program is built threaded? I think I see a deadlock condition that can occur in it if that's the case. > I do have PBS logs in /usr/spool/PBS/server_logs and my globus-pbs.conf > points there: > > log_path=/usr/spool/PBS/server_logs > > > Any help would be highly appreciated. > > Cheers, > Vladimir
