This is strange.

After restarting galaxy to include the debug tool, all galaxy_slots values are correctly assigned (tested for BWA and GATK as rerun of original posted jobs)....

If you still need the output, let me know.

Best,

Geert

On 03/14/2014 03:25 PM, John Chilton wrote:
I am not sure I understand what you mean, but if you are setting the
job to run with a ppn of greater than one but GALAXY_SLOTS is being
evaluated as 1 then you have likely uncovered a bug - maybe in the
runner, your job configuration, or the GALAXY_SLOTS logic for
pbs/torque.

I have created a Galaxy tool for debugging the problem -
https://gist.github.com/jmchilton/9548516. If you could run in your
environment and assign it multiple cores it should spit out some
interesting information relevant to debugging the problem. It will
create Galaxy datasets corresponding to the runtime-computed Galaxy
slots, the contents of PBS_NODEFILE which Galaxy attempts to use to
compute Galaxy slots, and the full contents of your worker node
environment. This last one may contain sensitive information - but if
you could post the first two or send them to me directly it would
hopefully help debug the problem.

-John

On Wed, Mar 12, 2014 at 12:37 PM, Geert Vandeweyer
<geert.vandewey...@uantwerpen.be> wrote:
How would those statements translate to pbs/torque (pbs_python)

I request resources using the nodes=1:ppn:4 syntax.  The -pe argument is not
available it seems...

Best,

Geert


On 03/12/2014 04:43 PM, Björn Grüning wrote:
Hi Geert,

you can give every tool a different amount of SLOTS via the job_conf.xml
file.

https://wiki.galaxyproject.org/Admin/Config/Jobs

For example:

         <tool
id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator"
destination="12cores_24G" />


where 12cores_24G is defined with a parallel SGE environment.

Cheers,
Bjoern

Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
Hi,

Is there documentation for the proper setup and utilisation of the
\${GALAXY_SLOTS:-4} style options?

I noticed that tools with this setting, including BWA and GATK run
single threaded, with the following settings respectively:

BWA  \${GALAXY_SLOTS:-4}"
===
ps output: python

/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
--threads=1 --fileSource=indexed
--ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
--do_not_build_index
--input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
--input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
--output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
--genAlignType=paired --params=pre_set --suppressHeader=false

xml config: --threads="\${GALAXY_SLOTS:-4}"


GATK Base Recalibrator \${GALAXY_SLOTS:-8}
==============
python

/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
--max_jvm_heap 6g --stdout
/galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
/galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
gatk_input -d
/galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat
bam_index
gatk_input -p           java -jar
"$GATK2_PATH/GenomeAnalysisTK.jar"        -T "BaseRecalibrator"
$GATK2_SITE_OPTIONS      --num_cpu_threads_per_data_thread
${GALAXY_SLOTS:-8}      --no_standard_covs          -R
"/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
"/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
"ContextCovariate"             -cov "CycleCovariate"     -d
--knownSites:dbsnp,%(file_type)s
/galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
/galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
"NONE" -p          --baq "OFF" --baqGapOpenPenalty
"40.0"                  --defaultBaseQualities "-1"
--validation_strictness "STRICT" --interval_merging "ALL"

java -Xmx6g -jar
/galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
-R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
--interval_set_rule UNION --downsampling_type NONE --baq OFF
--baqGapOpenPenalty 40.0 --defaultBaseQualities -1
--validation_strictness STRICT --interval_merging ALL -I
/tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
/tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
/tmp/tmp-gatk-fIuncH/input_intervals_0.bed


Best,

Geert





--

Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


--

Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Reply via email to