Hi.

Not really, I think. If you look here <https://www.biostars.org/p/63816/>,  Ole
introduces Gnu parallel and makes some example. One of these is the "*Blast
on multiple machines*" example and here he shows how to run blast under GNU
parallel first on the local node and then on multiple nodes.

I am trying to do the same stuff on our cluster so I have to submit that
single parallel command with bsub.

Anyhow I finally realized I can forget about blaunch and use simple ssh
instead (blaunch is kerberos compliant but I don't need it outside afs
partitions).


g




On Fri, Apr 17, 2015 at 4:20 PM, George Marselis <geo...@marsel.is> wrote:

> Giuseppe, too much unformatted shell to figure out what is happening at a
> glance.
>
> From what I can gather, you are using parallel to create scripts on
> different nodes, which then get submitted to LSF on that node they were
> created under.
>
> Am I right?
>
> On Thu, Apr 16, 2015 at 7:33 PM, Giuseppe Aprea <giuseppe.ap...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Maybe this time it could be a GNU parallel issue.
>>
>> I finally managed to run something with the following script:
>>
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>> #!/bin/bash
>>
>>
>> #BSUB -J gnuParallel_blast_test         # Name of the job.
>> #BSUB -o %J.out                         # Appends std output to file
>> %J.out. (%J is the Job ID)
>> #BSUB -e %J.err                         # Appends std error to file
>> %J.err.
>> #BSUB -q cresco3_h144                   # Queue name.
>> #BSUB -n 8                            # Number of CPUs.
>> #BSUB -W 1210 # walltime
>>
>> module load 4.8.3/ncbi/12.0.0
>> module load 4.8.3/parallel/20150122
>>
>> SLOTS=`cat ${LSB_DJOB_HOSTFILE} |wc -l`
>>
>> SERVER=""
>>
>> for i in `cat ${LSB_DJOB_HOSTFILE}| sort`
>> do
>> echo "/afs/enea.it/software/bin/blaunch.sh ${i}" >> servers
>> done
>>
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - >
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/all-vs-all-outfmt-6-filtered
>> cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000.fasta
>> | parallel --no-notice -vv -j ${SLOTS} --tmpdir
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp --wait
>> --slf servers --block 20k --recstart '>' --pipe blastp -evalue 1e-05
>> -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_{#}
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>>
>> The most important changes are
>>
>> - using --tmpdir options since temporary files must be on a shared
>> filesystem(!!).
>> - I used a larger data chunk (--block 20k instead of -N 1)
>> - I used --wait for LSF load easing
>> - I used {#} instead of {%} to tag result file for better debugging
>>
>> I finally got some non empty result files. Unfortunately I only got only
>> 8 file of expected 22
>>
>> these are stdout and stderr:
>>
>>
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par09c3l.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"3\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_3\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par26f1H.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"2\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_2\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/partdB11.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"5\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_5\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parzRBOV.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"8\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_8\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pardYnbH.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"9\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_9\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpLXXu.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"11\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_11\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parpDyze.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"12\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_12\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parska__.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"14\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_14\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par07hkH.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"15\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_15\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parwWxw0.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"17\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_17\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/paroNaAJ.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"18\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_18\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parchOGP.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"20\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_20\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parVhSi8.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"21\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_21\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pareLiHn.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"23\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_23\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parse3wL.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"6\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_6\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parQh5eJ.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x003.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"27\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_27\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parnuokL.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"25\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_25\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parmXJ5N.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"26\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_26\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parXM76W.chr;
>> cat - ) | (/afs/enea.it/software/bin/blaunch.sh
>> cresco3x020.portici.enea.it exec perl -e
>> \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"22663\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"24\\\"\\\;\\\$bashfunc\\\
>> =\\\ \\\"\\\"\\\;@ARGV=\\\"blastp\\\ -evalue\\\ 1e-05\\\ -outfmt\\\ 6\\\
>> -db\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000\\\
>> -query\\\ -\\\ -out\\\
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_24\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\););
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/pargNN8b.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_10);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parf3mob.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_7);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parGTSb6.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_16);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parxwGS2.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_4);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parCrFM_.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_1);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/par7fOlu.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_19);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parKlqXO.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_22);
>>  sh -c 'dd bs=1 count=1
>> of=/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr
>> 2>/dev/null';  test ! -s
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr"
>> && rm -f
>> "/gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr"
>> && exec true;  (cat
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr;
>> rm
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/tmp/parsuHqk.chr;
>> cat - ) | (blastp -evalue 1e-05 -outfmt 6 -db
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/goodProteins_first_0010000
>> -query - -out
>> /gporq1_1M/usr/aprea/bio/solanum_melongena/analysis/orthomcl_00/resultd_13);
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>>
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>> Apr 16 17:47:26 2015 22805 3 7.03 lsb_launch(): Failed while executing
>> tasks.
>> Apr 16 17:47:27 2015 22885 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22938 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22939 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22924 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:26 2015 22802 3 7.03 lsb_launch(): Failed while executing
>> tasks.
>> Apr 16 17:47:27 2015 22928 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22946 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22950 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22923 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22951 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22952 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22915 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 23022 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 22787 3 7.03 lsb_launch(): Failed while executing
>> tasks.
>> Apr 16 17:47:27 2015 23046 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 23054 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 23058 3 7.03 lsb_launch(): Connection refused by
>> server.
>> Apr 16 17:47:27 2015 23053 3 7.03 lsb_launch(): Connection refused by
>> server.
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> the server file is
>>
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>> /afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x003.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
>> /afs/enea.it/software/bin/blaunch.sh cresco3x020.portici.enea.it
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> And the master node is cresco3x003
>>
>> I don't understand the output from -vv options very well but, as you can
>> see (I hope) the last 8 lines from stdout (with indices
>> 1,4,7,10,13,16,19,22) are different and they correspond to the 8 output
>> file I finally retrieve. I would have expected the same lines at least for
>> the first 5 indices (1,2,3,4,5) since PARALLEL eventually could have
>> recognized it was on the local host but that's not the case and I find that
>> strange....
>>
>> Does anybody have an idea of what's going on?
>>
>> Ciao,
>>
>> Giuseppe
>>
>> PS I remind I am using version 20150122
>>
>>
>>
>>
>> On Thu, Apr 16, 2015 at 3:49 PM, George Marselis <geo...@marsel.is>
>> wrote:
>>
>>> You could do that, and it might work, if you are interested in using the
>>> LSF API
>>>
>>> otherwise, use a batch script and blaunch < batchscript.sh , instead.
>>>
>>> Ciao,
>>>
>>> George
>>>
>>> On Thu, Apr 16, 2015 at 9:24 AM, Giuseppe Aprea <
>>> giuseppe.ap...@gmail.com> wrote:
>>>
>>>> Dear George,
>>>>
>>>> I was not sure you were talking to me because I am submitting 1 job
>>>> only. Then I am using blaunch as a spawn agent instead of shh because this
>>>> is the way our cluster works. Mpi jobs with several hundreds slots also use
>>>> it (orte_rsh_agent MCA parameter).
>>>>
>>>> What do you think if I replace
>>>>
>>>> --recstart '>' -N 1 --pipe
>>>>
>>>> with
>>>>
>>>> --recstart '>' -block 100k --pipe
>>>>
>>>> ?
>>>>
>>>> I think I could reduce load on lsb_launch() using a larger data chunk,
>>>> couldn't I?
>>>>
>>>> cheers,
>>>>
>>>> g
>>>>
>>>>
>>>>
>>>> On Wed, Apr 15, 2015 at 10:32 PM, George Marselis <geo...@marsel.is>
>>>> wrote:
>>>>
>>>>> > You could use parallel to submit jobs, but its a very bad idea, due
>>>>> to the limitations of the software.
>>>>>
>>>>>
>>>>> and by that, I mean the limitations of the LSF software.
>>>>>
>>>>> Parallel rocks. Ole rocks.
>>>>>
>>>>> Next Parallel Release should be named "GNU Terry Pratchett"
>>>>>
>>>>> > Let me know which option is better for you.
>>>>> > As with regard to Martin, he should not use parallel for
>>>>>
>>>>> ^-- this is your brain on Perl.
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> George
>>>>>
>>>>>
>>>>> On Wed, Apr 15, 2015 at 11:27 PM, George Marselis <geo...@marsel.is>
>>>>> wrote:
>>>>>
>>>>>> Giuseppe, I was referring to both of you. My apologies I was not
>>>>>> clear, I had my head stuck in Perl while writing the first email.
>>>>>>
>>>>>> My suggestion to both of you is that you should not use parallel for
>>>>>> your respective topics.
>>>>>>
>>>>>> Giuseppe,
>>>>>>
>>>>>> You should use an extra script. Your problem is that you are timing
>>>>>> out while trying to submit all those jobs. The timeout happens because of
>>>>>> the number of jobs you are submitting: LSF cannot write the job
>>>>>> descriptions fast enough to disk, times out because the action is not
>>>>>> completed and then stays in that state
>>>>>>
>>>>>> ----------------
>>>>>> Martin,
>>>>>>
>>>>>> You could use parallel to submit jobs, but its a very bad idea, due
>>>>>> to the limitations of the software. Use batch scripts and job arrays when
>>>>>> possible.
>>>>>>
>>>>>> ----------------
>>>>>>
>>>>>> So, as per my suggestion, I think our discussion is offtopic for this
>>>>>> list. We could continue here, if Ole and the list puts up with us, but I
>>>>>> think we should take this on a personal email or switch this to the 
>>>>>> Debian
>>>>>> Medical email list https://en.wikipedia.org/wiki/Debian-Med .
>>>>>>
>>>>>> Let me know which option is better for you.
>>>>>>
>>>>>> As with regard to Martin, he should not use parallel for
>>>>>>
>>>>>>
>>>>>> Ciao,
>>>>>>
>>>>>> George
>>>>>>
>>>>>> On Wed, Apr 15, 2015 at 10:50 PM, Giuseppe Aprea <
>>>>>> giuseppe.ap...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi George!
>>>>>>>
>>>>>>> I am not sure who you are talking with. Martin or me? I remind the
>>>>>>> original topic is about using blast under parallel with LSF.
>>>>>>> Martin's problem sounds like something offtopic.
>>>>>>>
>>>>>>> You have both sysadmin and bioinformatics experience so I would
>>>>>>> really appreciate your help!
>>>>>>>
>>>>>>> I am working on a cluster so I must use LSF to get slots and I would
>>>>>>> prefer using parallel also since it splits input automatically with
>>>>>>> --recstart (which is quite nice:D otherwise I have to use another script
>>>>>>> for that). I see I could do better with chunksize (I have 1 record at 
>>>>>>> time
>>>>>>> in my example) but that's a secondary problem now. First I have the
>>>>>>> "lsb_launch(): Failed while waiting for tasks to finish." issue to 
>>>>>>> solve.
>>>>>>>
>>>>>>> cheers,
>>>>>>>
>>>>>>> g
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 15, 2015 at 7:44 PM, George Marselis <geo...@marsel.is>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> By the way, LSF and GNU parallel do almost the same thing. So using
>>>>>>>> one of the two, defeats the purpose of using the other.
>>>>>>>>
>>>>>>>> In the same way, you could have used LSF to submit your jobs to LSF:
>>>>>>>>
>>>>>>>> bsub < script.sh
>>>>>>>>
>>>>>>>> where script.sh was
>>>>>>>>
>>>>>>>> bsub -J amoeba -q smalljobs  qfasta file1
>>>>>>>> bsub -J amoeba -q smalljobs  qfasta file2
>>>>>>>> ...
>>>>>>>> bsub -J amoeba -q smalljobs  qfasta file2000
>>>>>>>>
>>>>>>>> On Wed, Apr 15, 2015 at 8:39 PM, George Marselis <geo...@marsel.is>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi. LSF/Openlava sysadmin in bioinformatics and parallel user here.
>>>>>>>>>
>>>>>>>>> I have seen this a couple more times: You are trying to use GNU
>>>>>>>>> parallel to submit the jobs to all nodes.
>>>>>>>>>
>>>>>>>>> THat's now the way to do things: You should not submit jobs on
>>>>>>>>> *all* your nodes. Please don't do that, as bsub was not designed to 
>>>>>>>>> read
>>>>>>>>> large chunks of jobs. bsub writes the jobs to your home directory, so 
>>>>>>>>> if
>>>>>>>>> your storage is not designed for a lot of writes, you are going to 
>>>>>>>>> blow the
>>>>>>>>> cluster out of the water.
>>>>>>>>>
>>>>>>>>> What you want to do is look up either:
>>>>>>>>>
>>>>>>>>> 1. bsub scripts
>>>>>>>>> https://rc.fas.harvard.edu/resources/documentation/legacy-lsf/lsf-submit-an-lsf-job/
>>>>>>>>>
>>>>>>>>> or
>>>>>>>>>
>>>>>>>>> 2. job arrays
>>>>>>>>> https://rc.fas.harvard.edu/resources/documentation/legacy-lsf/lsf-submitting-lots-of-short-jobs-job-arrays/
>>>>>>>>>
>>>>>>>>> Both bsub scripts and job arrays are useful to you: bsub scripts
>>>>>>>>> can be submitted as part of a pipeline: you can program the output of 
>>>>>>>>> the
>>>>>>>>> bsub script from your pipeline and then submit it to bsub. So, 
>>>>>>>>> instead of
>>>>>>>>> submitting your job 2000 times as in
>>>>>>>>>
>>>>>>>>> bsub job0
>>>>>>>>> bsub job1
>>>>>>>>>
>>>>>>>>> ....
>>>>>>>>>
>>>>>>>>> bsub job1999
>>>>>>>>>
>>>>>>>>> you just submit "bsub < scriptname" which contains 2000 lines
>>>>>>>>> which describe your jobs and you are done. The rest is done by 
>>>>>>>>> bsub/LSF
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Now, if your jobs are similar in a way that you just increment
>>>>>>>>> counter (as in most bioinformatics jobs), use arrays.
>>>>>>>>>
>>>>>>>>> bsub -J JOBNAME[0-1999], where JOBNAME is a string you would like
>>>>>>>>> to name your job as, eg "fasta files alignment"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> These techniques are useful because you can submit all 2000 jobs
>>>>>>>>> in less than a second, you can do it from a single node and you will 
>>>>>>>>> not
>>>>>>>>> have to deal with a grumpy sysadmin or grumpy colleagues who cannot 
>>>>>>>>> use the
>>>>>>>>> cluster. Just make sure you use the appropriate queue.
>>>>>>>>>
>>>>>>>>> Let me know if you have any questions.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>>
>>>>>>>>> George Marselis
>>>>>>>>>
>>>>>>>>> On Wed, Apr 15, 2015 at 6:48 PM, Martin d'Anjou <
>>>>>>>>> martin.danjo...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>  Hi,
>>>>>>>>>>
>>>>>>>>>> Thanks for clarifying. I want to use GNU Parallel to bsub jobs.
>>>>>>>>>> This way I can use GNU Parallel to throttle the number of jobs that 
>>>>>>>>>> are
>>>>>>>>>> submitted to LSF, and it is easier than writing a loop.
>>>>>>>>>>
>>>>>>>>>> parallel -j 100 my_script [bsub options] ::: {1..2000}
>>>>>>>>>>
>>>>>>>>>> my_script (pseudo-code):
>>>>>>>>>> #!/bin/bash
>>>>>>>>>> ...
>>>>>>>>>> bsub [bsub options] command ...
>>>>>>>>>> post-process data
>>>>>>>>>>
>>>>>>>>>> This way I can submit jobs, say 100 at a time. When I submit all
>>>>>>>>>> 2000 jobs, it gets problematic and I start hitting limits with file
>>>>>>>>>> descriptors, etc.
>>>>>>>>>>
>>>>>>>>>> Thanks for sharing,
>>>>>>>>>> Martin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 15-04-15 11:35 AM, Giuseppe Aprea wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Martin,
>>>>>>>>>>
>>>>>>>>>>  I am not sure I understand. As far as I can see, things work
>>>>>>>>>> exactly the opposite way: you have an LSF script which launches GNU
>>>>>>>>>> Parallel on some hosts provided by LSF. Something like:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>>> #!/bin/bash
>>>>>>>>>>
>>>>>>>>>>  #BSUB -J gnuParallel_blast_test      # Name of the job.
>>>>>>>>>> #BSUB -o %J.out                              # Appends std output
>>>>>>>>>> to file %J.out. (%J is the Job ID)
>>>>>>>>>> #BSUB -e %J.err                               # Appends std error
>>>>>>>>>> to file %J.err.
>>>>>>>>>> #BSUB -q large                                 # Queue name.
>>>>>>>>>> #BSUB -n 30                                      # Number of CPUs.
>>>>>>>>>>
>>>>>>>>>>  module load 4.8.3/ncbi/12.0.0
>>>>>>>>>> module load 4.8.3/parallel/20150122
>>>>>>>>>>
>>>>>>>>>>  SLOTS=`cat ${LSB_DJOB_HOSTFILE} |wc -l`
>>>>>>>>>>
>>>>>>>>>>  SERVER=""
>>>>>>>>>>
>>>>>>>>>>  for i in `cat ${LSB_DJOB_HOSTFILE}| sort`
>>>>>>>>>>  do
>>>>>>>>>>  echo "/afs/enea.it/software/bin/blaunch.sh ${i}" >> servers
>>>>>>>>>> done
>>>>>>>>>>
>>>>>>>>>>  cat absolute_path_to_sequences.fasta | parallel --no-notice -vv
>>>>>>>>>> -j ${SLOTS} --slf servers --plain --recstart '>' -N 1 --pipe blastp 
>>>>>>>>>> -evalue
>>>>>>>>>> 1e-05 -outfmt 6 -db absolute_path_to_db_file -query - -out
>>>>>>>>>> absolute_path_to_result_file_{%}
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>>  LSF is the one which gives you the execution hosts so if you
>>>>>>>>>> are launching bsub from GNU parallel how do you know how to set the 
>>>>>>>>>> --slf
>>>>>>>>>> option?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  g
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>   On Wed, Apr 15, 2015 at 4:24 PM, Martin d'Anjou <
>>>>>>>>>> martin.danjo...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> On 15-04-15 09:34 AM, Giuseppe Aprea wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> I would like to ask you, please, some help in using parallel
>>>>>>>>>>>> with blast alignment software.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I am trying to use GNU parallel v. 20150122 with blast for a
>>>>>>>>>>>> very large sequences alignment. I am using Parallel on a cluster 
>>>>>>>>>>>> which uses
>>>>>>>>>>>> LSF as queue system.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  Hello Giuseppe,
>>>>>>>>>>>
>>>>>>>>>>> I am an avid LSF user, and I want to use GNU Parallel to
>>>>>>>>>>> dispatch jobs to LSF. Could you please explain a little bit to me 
>>>>>>>>>>> how GNU
>>>>>>>>>>> Parallel works with LSF? I do not see it in the on-line tutorials. 
>>>>>>>>>>> For
>>>>>>>>>>> example, I would like to understand how to pass "bsub" options like 
>>>>>>>>>>> -oo, -q
>>>>>>>>>>> queue_name, etc. to LSF from GNU Parallel.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Martin
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Reply via email to