Hi List,

I'm having this same issue - if I dispatch a job to dynamic job runner, and it 
then hands the job to pbs, then it will flip out if I restart the server. In my 
case, though, there is no error in the job state - it is sitting in the Torque 
queue where it should be. As far as I can tell, galaxy successfully reloads the 
job back into the history. But it falters on recovering this destination for 
pbs.

If I delete the queued jobs, it quiets down, but this is not a sustainable 
solution - I need to be able to reboot the server without all of my users' 
cluster jobs causing faults every second.

Log:
galaxy.jobs.runners.pbs DEBUG 2013-10-04 15:50:32,454 Set default PBS server to 
m1.mason.indiana.edu
galaxy.jobs.runners DEBUG 2013-10-04 15:50:32,455 Starting 4 PBSRunner workers
galaxy.jobs DEBUG 2013-10-04 15:50:32,459 Loaded job runner 
'galaxy.jobs.runners.pbs:PBSJobRunner' as 'pbs'
galaxy.jobs.handler DEBUG 2013-10-04 15:50:32,459 Loaded job runners plugins: 
local:pbs
galaxy.jobs.handler INFO 2013-10-04 15:50:32,460 job handler stop queue started
galaxy.jobs DEBUG 2013-10-04 15:50:32,478 (514) Working directory for job is: 
/N/dc/projects/galaxy/job_working_directory/000/514
galaxy.jobs.handler DEBUG 2013-10-04 15:50:32,478 recovering job 514 in pbs 
runner
galaxy.jobs WARNING 2013-10-04 15:50:32,478 (514) Job runner URLs are 
deprecated, use destinations instead.
galaxy.jobs.runners.pbs DEBUG 2013-10-04 15:50:32,479 (514/176938.m1.mason) is 
still in PBS queued state, adding to the PBS queue
galaxy.jobs.handler INFO 2013-10-04 15:50:32,485 job handler queue started
...
galaxy.jobs.runners ERROR 2013-10-04 15:50:33,456 Unhandled exception checking 
active jobs
Traceback (most recent call last):
  File "/N/hd03/galaxy/Mason/galaxy-ncgas/lib/galaxy/jobs/runners/__init__.py", 
line 362, in monitor
    self.check_watched_items()
  File "/N/hd03/galaxy/Mason/galaxy-ncgas/lib/galaxy/jobs/runners/pbs.py", line 
385, in check_watched_items
    ( failures, statuses ) = self.check_all_jobs()
  File "/N/hd03/galaxy/Mason/galaxy-ncgas/lib/galaxy/jobs/runners/pbs.py", line 
466, in check_all_jobs
    pbs_server_name = 
self.__get_pbs_server(pbs_job_state.job_destination.params)
  File "/N/hd03/galaxy/Mason/galaxy-ncgas/lib/galaxy/jobs/runners/pbs.py", line 
222, in __get_pbs_server
    return job_destination_params['destination'].split('@')[-1]
KeyError: 'destination'


Thanks so much for wisdom on this.

Sincerely,

Carrie Ganote
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to