Hi,
Maybe this time it could be a GNU parallel issue.
I finally managed to run something with the following script:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
#BSUB -J gnuParallel_blast_test # Name of the job.
#BSUB -o %J.out # Appends std output to file
%J.out. (%J is the Job ID)
#BSUB -e %J.err # Appends std error to file %J.err.
#BSUB -q cresco3_h144 # Queue name.
#BSUB -n 8 # Number of CPUs.
#BSUB -W 1210 # walltime
module load 4.8.3/ncbi/12.0.0
module load 4.8.3/parallel/20150122
SLOTS=`cat ${LSB_DJOB_HOSTFILE} |wc -l`
SERVER=""
for i in `cat ${LSB_DJOB_HOSTFILE}| sort`
do
echo "/afs/enea.it/software/bin/blaunch.sh ${i}" >> servers
done
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - >
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/all-vs-all-outfmt-6-filtered
cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000.fasta
| parallel --no-notice -vv -j ${SLOTS} --tmpdir
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp --wait
--slf servers --block 20k --recstart '>' --pipe blastp -evalue 1e-05
-outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_{#}
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
The most important changes are
- using --tmpdir options since temporary files must be on a shared
filesystem(!!).
- I used a larger data chunk (--block 20k instead of -N 1)
- I used --wait for LSF load easing
- I used {#} instead of {%} to tag result file for better debugging
I finally got some non empty result files. Unfortunately I only got only 8
file of expected 22
these are stdout and stderr:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"3\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_3\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"2\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_2\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"5\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_5\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"8\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_8\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"9\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_9\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"11\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_11\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"12\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_12\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"14\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_14\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"15\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_15\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"17\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_17\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"18\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_18\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"20\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_20\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"21\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_21\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"23\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_23\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"6\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_6\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"27\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_27\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"25\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_25\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"26\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_26\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr;
cat - ) | (/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
exec perl -e
\\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"24\\\"\\\;\\\$bashfunc\\\
=\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
-db\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
-query\\\ -\\\ -out\\\
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_24\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_10);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_7);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_16);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_4);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_1);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_19);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_22);
sh -c 'dd bs=1 count=1
of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr
2>/dev/null'; test ! -s
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr"
&& rm -f
"/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr"
&& exec true; (cat
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr;
rm
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr;
cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
-query - -out
/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_13);
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Apr 16 17:47:26 2015 22805 3 7.03 lsb_launch(): Failed while executing
tasks.
Apr 16 17:47:27 2015 22885 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22938 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22939 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22924 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:26 2015 22802 3 7.03 lsb_launch(): Failed while executing
tasks.
Apr 16 17:47:27 2015 22928 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22946 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22950 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22923 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22951 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22952 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22915 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 23022 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 22787 3 7.03 lsb_launch(): Failed while executing
tasks.
Apr 16 17:47:27 2015 23046 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 23054 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 23058 3 7.03 lsb_launch(): Connection refused by
server.
Apr 16 17:47:27 2015 23053 3 7.03 lsb_launch(): Connection refused by
server.
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
the server file is
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
/afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------
And the master node is cresco3x003
I don't understand the output from -vv options very well but, as you can
see (I hope) the last 8 lines from stdout (with indices
1,4,7,10,13,16,19,22) are different and they correspond to the 8 output
file I finally retrieve. I would have expected the same lines at least for
the first 5 indices (1,2,3,4,5) since PARALLEL eventually could have
recognized it was on the local host but that's not the case and I find that
strange....
Does anybody have an idea of what's going on?
Ciao,
Giuseppe
PS I remind I am using version 20150122
On Thu, Apr 16, 2015 at 3:49 PM, George Marselis <geo...@marsel.is> wrote:
> You could do that, and it might work, if you are interested in using the
> LSF API
>
> otherwise, use a batch script and blaunch < batchscript.sh , instead.
>
> Ciao,
>
> George
>
> On Thu, Apr 16, 2015 at 9:24 AM, Giuseppe Aprea <giuseppe.ap...@gmail.com>
> wrote:
>
>> Dear George,
>>
>> I was not sure you were talking to me because I am submitting 1 job only.
>> Then I am using blaunch as a spawn agent instead of shh because this is the
>> way our cluster works. Mpi jobs with several hundreds slots also use it (
>> orte_rsh_agent MCA parameter).
>>
>> What do you think if I replace
>>
>> --recstart '>' -N 1 --pipe
>>
>> with
>>
>> --recstart '>' -block 100k --pipe
>>
>> ?
>>
>> I think I could reduce load on lsb_launch() using a larger data chunk,
>> couldn't I?
>>
>> cheers,
>>
>> g
>>
>>
>>
>> On Wed, Apr 15, 2015 at 10:32 PM, George Marselis <geo...@marsel.is>
>> wrote:
>>
>>> > You could use parallel to submit jobs, but its a very bad idea, due to
>>> the limitations of the software.
>>>
>>>
>>> and by that, I mean the limitations of the LSF software.
>>>
>>> Parallel rocks. Ole rocks.
>>>
>>> Next Parallel Release should be named "GNU Terry Pratchett"
>>>
>>> > Let me know which option is better for you.
>>> > As with regard to Martin, he should not use parallel for
>>>
>>> ^-- this is your brain on Perl.
>>>
>>> Best Regards,
>>>
>>> George
>>>
>>>
>>> On Wed, Apr 15, 2015 at 11:27 PM, George Marselis <geo...@marsel.is>
>>> wrote:
>>>
>>>> Giuseppe, I was referring to both of you. My apologies I was not clear,
>>>> I had my head stuck in Perl while writing the first email.
>>>>
>>>> My suggestion to both of you is that you should not use parallel for
>>>> your respective topics.
>>>>
>>>> Giuseppe,
>>>>
>>>> You should use an extra script. Your problem is that you are timing out
>>>> while trying to submit all those jobs. The timeout happens because of the
>>>> number of jobs you are submitting: LSF cannot write the job descriptions
>>>> fast enough to disk, times out because the action is not completed and then
>>>> stays in that state
>>>>
>>>> ----------------
>>>> Martin,
>>>>
>>>> You could use parallel to submit jobs, but its a very bad idea, due to
>>>> the limitations of the software. Use batch scripts and job arrays when
>>>> possible.
>>>>
>>>> ----------------
>>>>
>>>> So, as per my suggestion, I think our discussion is offtopic for this
>>>> list. We could continue here, if Ole and the list puts up with us, but I
>>>> think we should take this on a personal email or switch this to the Debian
>>>> Medical email list https://en.wikipedia.org/wiki/Debian-Med .
>>>>
>>>> Let me know which option is better for you.
>>>>
>>>> As with regard to Martin, he should not use parallel for
>>>>
>>>>
>>>> Ciao,
>>>>
>>>> George
>>>>
>>>> On Wed, Apr 15, 2015 at 10:50 PM, Giuseppe Aprea <
>>>> giuseppe.ap...@gmail.com> wrote:
>>>>
>>>>> Hi George!
>>>>>
>>>>> I am not sure who you are talking with. Martin or me? I remind the
>>>>> original topic is about using blast under parallel with LSF.
>>>>> Martin's problem sounds like something offtopic.
>>>>>
>>>>> You have both sysadmin and bioinformatics experience so I would really
>>>>> appreciate your help!
>>>>>
>>>>> I am working on a cluster so I must use LSF to get slots and I would
>>>>> prefer using parallel also since it splits input automatically with
>>>>> --recstart (which is quite nice:D otherwise I have to use another script
>>>>> for that). I see I could do better with chunksize (I have 1 record at time
>>>>> in my example) but that's a secondary problem now. First I have the
>>>>> "lsb_launch(): Failed while waiting for tasks to finish." issue to solve.
>>>>>
>>>>> cheers,
>>>>>
>>>>> g
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Apr 15, 2015 at 7:44 PM, George Marselis <geo...@marsel.is>
>>>>> wrote:
>>>>>
>>>>>> By the way, LSF and GNU parallel do almost the same thing. So using
>>>>>> one of the two, defeats the purpose of using the other.
>>>>>>
>>>>>> In the same way, you could have used LSF to submit your jobs to LSF:
>>>>>>
>>>>>> bsub < script.sh
>>>>>>
>>>>>> where script.sh was
>>>>>>
>>>>>> bsub -J amoeba -q smalljobs qfasta file1
>>>>>> bsub -J amoeba -q smalljobs qfasta file2
>>>>>> ...
>>>>>> bsub -J amoeba -q smalljobs qfasta file2000
>>>>>>
>>>>>> On Wed, Apr 15, 2015 at 8:39 PM, George Marselis <geo...@marsel.is>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi. LSF/Openlava sysadmin in bioinformatics and parallel user here.
>>>>>>>
>>>>>>> I have seen this a couple more times: You are trying to use GNU
>>>>>>> parallel to submit the jobs to all nodes.
>>>>>>>
>>>>>>> THat's now the way to do things: You should not submit jobs on *all*
>>>>>>> your nodes. Please don't do that, as bsub was not designed to read large
>>>>>>> chunks of jobs. bsub writes the jobs to your home directory, so if your
>>>>>>> storage is not designed for a lot of writes, you are going to blow the
>>>>>>> cluster out of the water.
>>>>>>>
>>>>>>> What you want to do is look up either:
>>>>>>>
>>>>>>> 1. bsub scripts
>>>>>>> https://rc.fas.harvard.edu/resources/documentation/legacy-lsf/lsf-submit-an-lsf-job/
>>>>>>>
>>>>>>> or
>>>>>>>
>>>>>>> 2. job arrays
>>>>>>> https://rc.fas.harvard.edu/resources/documentation/legacy-lsf/lsf-submitting-lots-of-short-jobs-job-arrays/
>>>>>>>
>>>>>>> Both bsub scripts and job arrays are useful to you: bsub scripts can
>>>>>>> be submitted as part of a pipeline: you can program the output of the
>>>>>>> bsub
>>>>>>> script from your pipeline and then submit it to bsub. So, instead of
>>>>>>> submitting your job 2000 times as in
>>>>>>>
>>>>>>> bsub job0
>>>>>>> bsub job1
>>>>>>>
>>>>>>> ....
>>>>>>>
>>>>>>> bsub job1999
>>>>>>>
>>>>>>> you just submit "bsub < scriptname" which contains 2000 lines which
>>>>>>> describe your jobs and you are done. The rest is done by bsub/LSF
>>>>>>>
>>>>>>>
>>>>>>> Now, if your jobs are similar in a way that you just increment
>>>>>>> counter (as in most bioinformatics jobs), use arrays.
>>>>>>>
>>>>>>> bsub -J JOBNAME[0-1999], where JOBNAME is a string you would like
>>>>>>> to name your job as, eg "fasta files alignment"
>>>>>>>
>>>>>>>
>>>>>>> These techniques are useful because you can submit all 2000 jobs in
>>>>>>> less than a second, you can do it from a single node and you will not
>>>>>>> have
>>>>>>> to deal with a grumpy sysadmin or grumpy colleagues who cannot use the
>>>>>>> cluster. Just make sure you use the appropriate queue.
>>>>>>>
>>>>>>> Let me know if you have any questions.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>>
>>>>>>> George Marselis
>>>>>>>
>>>>>>> On Wed, Apr 15, 2015 at 6:48 PM, Martin d'Anjou <
>>>>>>> martin.danjo...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Thanks for clarifying. I want to use GNU Parallel to bsub jobs.
>>>>>>>> This way I can use GNU Parallel to throttle the number of jobs that are
>>>>>>>> submitted to LSF, and it is easier than writing a loop.
>>>>>>>>
>>>>>>>> parallel -j 100 my_script [bsub options] ::: {1..2000}
>>>>>>>>
>>>>>>>> my_script (pseudo-code):
>>>>>>>> #!/bin/bash
>>>>>>>> ...
>>>>>>>> bsub [bsub options] command ...
>>>>>>>> post-process data
>>>>>>>>
>>>>>>>> This way I can submit jobs, say 100 at a time. When I submit all
>>>>>>>> 2000 jobs, it gets problematic and I start hitting limits with file
>>>>>>>> descriptors, etc.
>>>>>>>>
>>>>>>>> Thanks for sharing,
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>>> On 15-04-15 11:35 AM, Giuseppe Aprea wrote:
>>>>>>>>
>>>>>>>> Hi Martin,
>>>>>>>>
>>>>>>>> I am not sure I understand. As far as I can see, things work
>>>>>>>> exactly the opposite way: you have an LSF script which launches GNU
>>>>>>>> Parallel on some hosts provided by LSF. Something like:
>>>>>>>>
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>> #!/bin/bash
>>>>>>>>
>>>>>>>> #BSUB -J gnuParallel_blast_test # Name of the job.
>>>>>>>> #BSUB -o %J.out # Appends std output
>>>>>>>> to file %J.out. (%J is the Job ID)
>>>>>>>> #BSUB -e %J.err # Appends std error
>>>>>>>> to file %J.err.
>>>>>>>> #BSUB -q large # Queue name.
>>>>>>>> #BSUB -n 30 # Number of CPUs.
>>>>>>>>
>>>>>>>> module load 4.8.3/ncbi/12.0.0
>>>>>>>> module load 4.8.3/parallel/20150122
>>>>>>>>
>>>>>>>> SLOTS=`cat ${LSB_DJOB_HOSTFILE} |wc -l`
>>>>>>>>
>>>>>>>> SERVER=""
>>>>>>>>
>>>>>>>> for i in `cat ${LSB_DJOB_HOSTFILE}| sort`
>>>>>>>> do
>>>>>>>> echo "/afs/enea.it/software/bin/blaunch.sh ${i}" >> servers
>>>>>>>> done
>>>>>>>>
>>>>>>>> cat absolute_path_to_sequences.fasta | parallel --no-notice -vv
>>>>>>>> -j ${SLOTS} --slf servers --plain --recstart '>' -N 1 --pipe blastp
>>>>>>>> -evalue
>>>>>>>> 1e-05 -outfmt 6 -db absolute_path_to_db_file -query - -out
>>>>>>>> absolute_path_to_result_file_{%}
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> LSF is the one which gives you the execution hosts so if you are
>>>>>>>> launching bsub from GNU parallel how do you know how to set the --slf
>>>>>>>> option?
>>>>>>>>
>>>>>>>>
>>>>>>>> g
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 15, 2015 at 4:24 PM, Martin d'Anjou <
>>>>>>>> martin.danjo...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On 15-04-15 09:34 AM, Giuseppe Aprea wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I would like to ask you, please, some help in using parallel with
>>>>>>>>>> blast alignment software.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am trying to use GNU parallel v. 20150122 with blast for a very
>>>>>>>>>> large sequences alignment. I am using Parallel on a cluster which
>>>>>>>>>> uses LSF
>>>>>>>>>> as queue system.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hello Giuseppe,
>>>>>>>>>
>>>>>>>>> I am an avid LSF user, and I want to use GNU Parallel to dispatch
>>>>>>>>> jobs to LSF. Could you please explain a little bit to me how GNU
>>>>>>>>> Parallel
>>>>>>>>> works with LSF? I do not see it in the on-line tutorials. For
>>>>>>>>> example, I
>>>>>>>>> would like to understand how to pass "bsub" options like -oo, -q
>>>>>>>>> queue_name, etc. to LSF from GNU Parallel.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Martin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
