Hi,
What I remember is that while the lastz had died the python wrapper was still hanging around for
these 2 days. Maybe it's related to this builtin scheduler of the lastz wrapper.
Unfortunately I don't have time now to reproduce the error conditions.
regards, Andreas
On 26.11.2012 15:47, Bob Harris wrote:
Howdy, Andreas,
Lastz memory requirements are dependent on the size of the input sequences
(mainly on the size of
reference sequence) and, to a lesser extent, the genome's repeat content.
I'm a little confused/concerned by how this failure was indicated. When run
from a console, if
lastz has a memory allocation failure, a message is written to stderr (e.g.
"call to realloc failed
to allocate 1,000,000,000 bytes") and the program exits to the shell with the
status EXIT_FAILURE
(an ISO C definition which I presume corresponds to a standard shell error
code).
Usually, if lastz isn't going to have enough memory, an allocation failure will
occur early in the
run as all the long term data structures are being built. This would normally
be within the first
five minutes. A later failure would (probably) mean that the long term
structures were close to the
memory limit and then alignments for one of the query sequences required enough
additional memory to
push us over the limit.
I assume galaxy's "Job did not produce output" message must be based on a lack
of any output to
stdout. What is strange is that fact that it took 2 days to get this message.
Lack of output to
stdout suggests that the failure occurred before any queries were processed
(strictly speaking,
before any alignments were output). This should have occurred in the first few
minutes of the run.
Would it be possible for you to point me at the input sequences for this run,
so that I can try
running this via the console, and see if there's something happening in lastz
that I don't understand?
Bob H
On Nov 26, 2012, at 3:35 AM, Andreas Kuntzagk wrote:
Hi,
I just noticed that this was another case of a tool that needs more then the
1GB memory that is
default on our cluster. After adjusting the job_runner settings everything
seems fine.
regards, Andreas
On 15.11.2012 09:08, Andreas Kuntzagk wrote:
Hi,
when I start a lastz job it submits a job to GridEngine. But this job seems to
hang.
The process running is
python /data/galaxy/galaxy-dist/tools/sr_mapping/lastz_wrapper.py --ref_source=
...
This wrapper seems to not start a lastz but lastz is in the path.
After 2 days then Galaxy shows "Job did not produce output"
Here is the paster.log from the start of the job.
galaxy.jobs DEBUG 2012-11-15 09:03:07,539 (254) Working directory for job is:
/data/galaxy/galaxy-dist/database/job_working_directory/000/254
galaxy.jobs.handler DEBUG 2012-11-15 09:03:07,540 dispatching job 254 to drmaa
runner
galaxy.jobs.handler INFO 2012-11-15 09:03:07,688 (254) Job dispatched
galaxy.tools DEBUG 2012-11-15 09:03:07,980 Building dependency shell command
for dependency 'lastz'
galaxy.tools WARNING 2012-11-15 09:03:07,981 Failed to resolve dependency on
'lastz', ignoring
galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:08,669 (254) submitting file
/data/galaxy/galaxy-dist/database/pbs/galaxy_254.sh
galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:08,670 (254) command is: python
/data/galaxy/galaxy-dist/tools/sr_mapping/lastz_wrapper.py
--ref_source=cached
--source_select=pre_set --out_format=sam
--input2=/data/galaxy/galaxy-dist/database/files/027/dataset_27795.dat
--input1="/data/galaxy/galaxy-dist/tool-data/shared/ucsc/hg18/seq/hg18.2bit"
--ref_sequences="None" --pre_set_options=yasra98 --identity_min=0
--identity_max=100 --coverage=0
--output=/data/galaxy/galaxy-dist/database/job_working_directory/000/254/galaxy_dataset_27800.dat
--unmask=yes --lastzSeqsFileDir=/data/galaxy/galaxy-dist/tool-data; cd
/data/galaxy/galaxy-dist; /data/galaxy/galaxy-dist/set_metadata.sh
./database/files
/data/galaxy/galaxy-dist/database/job_working_directory/000/254 .
/data/galaxy/galaxy-dist/universe_wsgi.ini
/data/galaxy/galaxy-dist/database/tmp/tmpL4MMJV
/data/galaxy/galaxy-dist/database/job_working_directory/000/254/galaxy.json
/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_in_HistoryDatasetAssociation_288_9ynzEN,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_kwds_HistoryDatasetAssociation_288_AEIseO,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_out_HistoryDatasetAssociation_288_RhsufT,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_results_HistoryDatasetAssociation_288_86DYsR,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/galaxy_dataset_27800.dat,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_override_HistoryDatasetAssociation_288_HMRNEG
galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:08,676 run as user
['kuntzagk', '600']
galaxy.jobs DEBUG 2012-11-15 09:03:08,676 (254) Changing ownership of working
directory with:
/usr/bin/sudo -E scripts/external_chown_script.py
/data/galaxy/galaxy-dist/database/job_working_directory/000/254 kuntzagk 600
galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:09,148 (254) Job script for
external submission is:
/data/galaxy/galaxy-dist/database/pbs/254.jt_json
141.80.188.178 - - [15/Nov/2012:09:03:10 +0200] "POST
/galaxy/root/history_item_updates HTTP/1.1"
200 - "http://bbc.mdc-berlin.de/galaxy/history"; "Mozilla/5.0 (X11; Ubuntu;
Linux x86_64; rv:16.0)
Gecko/20100101 Firefox/16.0"
galaxy.jobs.runners.drmaa INFO 2012-11-15 09:03:11,553 (254) queued as 1282605
galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:11,767 (254/1282605) state
change: job is running
--
Andreas Kuntzagk
SystemAdministrator
Berlin Institute for Medical Systems Biology at the
Max-Delbrueck-Center for Molecular Medicine
Robert-Roessle-Str. 10, 13125 Berlin, Germany
http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
--
Andreas Kuntzagk
SystemAdministrator
Berlin Institute for Medical Systems Biology at the
Max-Delbrueck-Center for Molecular Medicine
Robert-Roessle-Str. 10, 13125 Berlin, Germany
http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
