More detailed command to try:
*#PBS -N LargeJob3
#PBS -o LargeJob3.stdout
#PBS -e LargeJob3.stderr
#PBS -l walltime=120:00:00
#PBS -l nodes=100
module load openmpi_gcc64/1.4.3_ofed
mpiexec -n 1024 \
Ray -k 23 -o **LargeJob3**\
-write-checkpoints \
-route-messages -connection-type debruijn -routing-graph-degree 32 \
-p pairs11.fastq pairs12.fastq \
-p pairs21.fastq pairs22.fastq \
-p pairs31.fastq pairs32.fastq \
-p mates11.fastq mates11.fastq \
-p mates21.fastq mates21.fastq *
100 nodes is a lot. It will give you collectively
100*32 giga-bytes of memory, or 3.2 tera-bytes.
This will use a de Bruijn graph of degree 32 for the communication.
This will reduce memory usage by Open-MPI and will also reduces a lot
the latency to something
near
mpiexec will allocate nodes with a round-robin policy by default I think.
Louis Letourneau a écrit :
~300MB genome
2 librarires of paires (hiseq2000) (2nd library on 2 lanes)
96499384 x 2
47407253 x 2
13776832 x 2
2 libraries of mates 5kb (done with GAIIx)
26300454 x 2
25851346 x 2
Total 419670538 reads, 41967053800 bases, 139x (I know it's a bit high)
On Mammouth, if you know the cluster, using 75 nodes, 10 of 24 cores
(because of ram, I cannot use all 24 cores).
120hours walltime is not enough
command (I removed file names and some PBS settings for brievety):
#PBS -l nodes=100
#PBS -l walltime=120:00:00
export ppn=10
mpiexec -n $[PBS_NUM_NODES*ppn] -npernode $ppn Ray-v2.0.0-rc8/Ray -k 23
-p pairs -p pairs -p pairs -p mates -p mates -o combined_23
-one-color-per-file -write-checkpoints> combined_23.out 2>&1
Louis
On 6/12/2012 2:47 PM, Sébastien Boisvert wrote:
Yes, it does that.
There will be binary files with the ".ray" extension in the
directory where you launched Ray.
You can not change the k-mer length when starting from old checkpoints.
The command needs to have the same number of arguments in the same order.
On what kind of dataset are you exceeding time limits ?
Louis Letourneau a écrit :
I saw these options on Ray
Checkpointing
-write-checkpoints
Write checkpoint files
-read-checkpoints
Read checkpoint files
-read-write-checkpoints
Read and write checkpoint files
I'm hitting walltimes on the cluster I'm using and I'm wondering if by
setting:
-read-write-checkpoints
I can resume where Ray got killed because of walltime?
If that's the purpose, what a great feature! :-)
Louis
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Denovoassembler-users mailing list
Denovoassembler-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Denovoassembler-users mailing list
Denovoassembler-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Denovoassembler-users mailing list
Denovoassembler-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
