On Thu, Nov 15, 2012 at 10:06 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote: > Hi all, > > Something has changed in the job handling, and in a bad way. On my > development machine submitting jobs to the cluster didn't seem to be > working anymore (never sent to SGE). I killed Galaxy and restarted: > ... > (segmentation fault)
Looking into the problem with submitting the jobs, there seems to be a problem with task splitting somehow recursing - the same file is split four times, the filename getting longer and longer: galaxy.jobs DEBUG 2012-11-15 10:08:33,510 (321) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/321 galaxy.jobs.handler DEBUG 2012-11-15 10:08:33,510 dispatching job 321 to tasks runner galaxy.jobs.handler INFO 2012-11-15 10:08:33,714 (321) Job dispatched galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:34,457 Split /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into batches of 1000 records... galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:34,457 Attemping to split FASTA file /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into chunks of 1000 sequences galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:34,458 Writing /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat part to /mnt/galaxy/galaxy-central/database/job_working_directory/000/321/task_0/dataset_344.dat galaxy.jobs.splitters.multi DEBUG 2012-11-15 10:08:34,458 do_split created 1 parts galaxy.jobs DEBUG 2012-11-15 10:08:34,558 (321) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/321 galaxy.jobs.handler DEBUG 2012-11-15 10:08:34,558 dispatching task 823, of job 321, to tasks runner 127.0.0.1 - - [15/Nov/2012:10:08:35 +0100] "POST /root/history_item_updates HTTP/1.1" 200 - "http://127.0.0.1:8081/history"; "Mozilla/5.0 (X11; Linux x86_64; rv:10.0.8) Gecko/20121012 Firefox/10.0.8" galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:35,458 Split /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into batches of 1000 records... galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:35,459 Attemping to split FASTA file /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into chunks of 1000 sequences galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:35,459 Writing /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat part to /mnt/galaxy/galaxy-central/database/job_working_directory/000/321/task_0/task_0/dataset_344.dat galaxy.jobs.splitters.multi DEBUG 2012-11-15 10:08:35,459 do_split created 1 parts galaxy.jobs DEBUG 2012-11-15 10:08:35,541 (321) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/321 galaxy.jobs.handler DEBUG 2012-11-15 10:08:35,542 dispatching task 824, of job 321, to tasks runner galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:36,171 Split /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into batches of 1000 records... galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:36,171 Attemping to split FASTA file /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into chunks of 1000 sequences galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:36,171 Writing /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat part to /mnt/galaxy/galaxy-central/database/job_working_directory/000/321/task_0/task_0/task_0/dataset_344.dat galaxy.jobs.splitters.multi DEBUG 2012-11-15 10:08:36,172 do_split created 1 parts galaxy.jobs DEBUG 2012-11-15 10:08:36,232 (321) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/321 galaxy.jobs.handler DEBUG 2012-11-15 10:08:36,232 dispatching task 825, of job 321, to tasks runner galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:36,843 Split /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into batches of 1000 records... galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:36,843 Attemping to split FASTA file /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat into chunks of 1000 sequences galaxy.datatypes.sequence DEBUG 2012-11-15 10:08:36,843 Writing /mnt/galaxy/galaxy-central/database/files/000/dataset_344.dat part to /mnt/galaxy/galaxy-central/database/job_working_directory/000/321/task_0/task_0/task_0/task_0/dataset_344.dat galaxy.jobs.splitters.multi DEBUG 2012-11-15 10:08:36,844 do_split created 1 parts galaxy.jobs DEBUG 2012-11-15 10:08:36,906 (321) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/321 galaxy.jobs.handler DEBUG 2012-11-15 10:08:36,906 dispatching task 826, of job 321, to tasks runner Hmm. Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
