> In the tests I did on this platform, I found that in general
> Ray needed between 400 MiB and 1 GiB of RAM per MPI rank.
We must be doing something wrong then, because we were crashing out with
debug messages indicating over 2 GB in some ranks at least:
Rank 130: assembler memory usage: 2247572 KiB
2013-11-05 13:56:11.462 (FATAL) [0xfffa41b8bb0]
844455:ibm.runjob.client.Job: terminated due to: killing the job timed
out
2013-11-05 13:56:11.463 (FATAL) [0xfffa41b8bb0]
844455:ibm.runjob.client.Job: abnormal termination by signal 11 from
rank 386. Delivering SIGKILL with status RUNNING timed out after 60
seconds
2013-11-05 13:56:11.464 (FATAL) [0xfffa41b8bb0]
844455:ibm.runjob.client.Job: 1 RAS event
2013-11-05 13:56:11.464 (FATAL) [0xfffa41b8bb0]
844455:ibm.runjob.client.Job: most recent RAS event text: killing job
844455 timed out after 60 seconds. 256 nodes are now unavailable.
Is there something in the attached configuration file that is allowing
such excess in memory utilization? We would appreciate any assistance
as our experience with Ray is very limited at this point.
Regards
--
Jeff Tan
High Performance Computing Specialist
IBM Research Collaboratory for Life Sciences, Melbourne
-k 31
-p Female_PE10_read1_sequence_trim.fastq Female_PE10_read2_sequence_trim.fastq
-p Female_PE11_read1_sequence_trim.fastq Female_PE11_read2_sequence_trim.fastq
-p Female_PE12_read1_sequence_trim.fastq Female_PE12_read2_sequence_trim.fastq
-p Female_PE13_read1_sequence_trim.fastq Female_PE13_read2_sequence_trim.fastq
-p Female_PE14_read1_sequence_trim.fastq Female_PE14_read2_sequence_trim.fastq
-p Female_PE15_read1_sequence_trim.fastq Female_PE15_read2_sequence_trim.fastq
-p Female_PE1_read1_sequence_trim.fastq Female_PE1_read2_sequence_trim.fastq
-p Female_PE2_read1_sequence_trim.fastq Female_PE2_read2_sequence_trim.fastq
-p Female_PE3_read1_sequence_trim.fastq Female_PE3_read2_sequence_trim.fastq
-p Female_PE4_read1_sequence_trim.fastq Female_PE4_read2_sequence_trim.fastq
-p Female_PE5_read1_sequence_trim.fastq Female_PE5_read2_sequence_trim.fastq
-p Female_PE6_read1_sequence_trim.fastq Female_PE6_read2_sequence_trim.fastq
-p Female_PE7_read1_sequence_trim.fastq Female_PE7_read2_sequence_trim.fastq
-p Female_PE8_read1_sequence_trim.fastq Female_PE8_read2_sequence_trim.fastq
-p Female_PE9_read1_sequence_trim.fastq Female_PE9_read2_sequence_trim.fastq
-p GM_260-280_2_1_sequence_trim.fastq GM_260-280_2_2_sequence_trim.fastq
-p GM_260-280_3_1_sequence_trim.fastq GM_260-280_3_2_sequence_trim.fastq
-p GM_260-280_4_1_sequence_trim.fastq GM_260-280_4_2_sequence_trim.fastq
-p GM_260-280_5_1_sequence_trim.fastq GM_260-280_5_2_sequence_trim.fastq
-p Set23GB_GM_260-280_2_1_sequence_trim.fastq
Set23GB_GM_260-280_2_2_sequence_trim.fastq
-p Set23GB_GM_260-280_3_1_sequence_trim.fastq
Set23GB_GM_260-280_3_2_sequence_trim.fastq
-p Set23GB_GM_260-280_4_1_sequence_trim.fastq
Set23GB_GM_260-280_4_2_sequence_trim.fastq
-p Set23GB_GM_260-280_5_1_sequence_trim.fastq
Set23GB_GM_260-280_5_2_sequence_trim.fastq
-p Set2_GM_260-280_2_1_sequence_trim.fastq
Set2_GM_260-280_2_2_sequence_trim.fastq
-p Set2_GM_260-280_3_1_sequence_trim.fastq
Set2_GM_260-280_3_2_sequence_trim.fastq
-p Set2_GM_260-280_4_1_sequence_trim.fastq
Set2_GM_260-280_4_2_sequence_trim.fastq
-p Set2_GM_260-280_5_1_sequence_trim.fastq
Set2_GM_260-280_5_2_sequence_trim.fastq
-p Set2_GM_260-280_6_1_sequence_trim.fastq
Set2_GM_260-280_6_2_sequence_trim.fastq
-p Set2_GM_260-280_7_1_sequence_trim.fastq
Set2_GM_260-280_7_2_sequence_trim.fastq
-p Set2_GM_260-280_8_1_sequence_trim.fastq
Set2_GM_260-280_8_2_sequence_trim.fastq
-p Male-PE-1_read1_sequence_trim.fastq Male-PE-1_read2_sequence_trim.fastq
-p Male-PE-2_read1_sequence_trim.fastq Male-PE-2_read2_sequence_trim.fastq
-p Male-PE-3_read1_sequence_trim.fastq Male-PE-3_read2_sequence_trim.fastq
-p Male-PE-4_read1_sequence_trim.fastq Male-PE-4_read2_sequence_trim.fastq
-p Male-PE-5_read1_sequence_trim.fastq Male-PE-5_read2_sequence_trim.fastq
-p Male-PE-6_read1_sequence_trim.fastq Male-PE-6_read2_sequence_trim.fastq
-p Male-PE-7_read1_sequence_trim.fastq Male-PE-7_read2_sequence_trim.fastq
-p Male-PE-8_read1_sequence_trim.fastq Male-PE-8_read2_sequence_trim.fastq
-p Male-PE-9_read1_sequence_trim.fastq Male-PE-9_read2_sequence_trim.fastq
-p PE1_500_read1_sequence_trim.fastq PE1_500_read2_sequence_trim.fastq
-p PE2_500_read1_sequence_trim.fastq PE2_500_read2_sequence_trim.fastq
-p PE3_500_read1_sequence_trim.fastq PE3_500_read2_sequence_trim.fastq
-p PE1_800_read1_sequence_trim.fastq PE1_800_read2_sequence_trim.fastq
-p PE2_800_read1_sequence_trim.fastq PE2_800_read2_sequence_trim.fastq
-p PE3_800_read1_sequence_trim.fastq PE3_800_read2_sequence_trim.fastq
-p PE4_800_read1_sequence_trim.fastq PE4_800_read2_sequence_trim.fastq
-p PE5_800_read1_sequence_trim.fastq PE5_800_read2_sequence_trim.fastq
-p PE6_800_read1_sequence_trim.fastq PE6_800_read2_sequence_trim.fastq
-p PE7_800_read1_sequence_trim.fastq PE7_800_read2_sequence_trim.fastq
-p PE8_800_read1_sequence_trim.fastq PE8_800_read2_sequence_trim.fastq
-p PE9_800_read1_sequence_trim.fastq PE9_800_read2_sequence_trim.fastq
-p MP1_2kb_read1_sequence_trim.fastq MP1_2kb_read2_sequence_trim.fastq 2000 112
-p MP2_2kb_read1_sequence_trim.fastq MP2_2kb_read2_sequence_trim.fastq 2000 197
-p MP3_2kb_read1_sequence_trim.fastq MP3_2kb_read2_sequence_trim.fastq 2000 196
-p Male-4kb-1_read1_sequence_trim.fastq Male-4kb-1_read2_sequence_trim.fastq
4000 112
-p Male-4kb-2_read1_sequence_trim.fastq Male-4kb-2_read2_sequence_trim.fastq
4000 111
-p Male-4kb-3_read1_sequence_trim.fastq Male-4kb-3_read2_sequence_trim.fastq
4000 110
-p Male-4kb-4_read1_sequence_trim.fastq Male-4kb-4_read2_sequence_trim.fastq
4000 110
-p Male-4kb-5_read1_sequence_trim.fastq Male-4kb-5_read2_sequence_trim.fastq
4000 111
-p MP1_4kb_read1_sequence_trim.fastq MP1_4kb_read2_sequence_trim.fastq 4000 96
-p MP2_4kb_read1_sequence_trim.fastq MP2_4kb_read2_sequence_trim.fastq 4000 107
-p MP3_4kb_read1_sequence_trim.fastq MP3_4kb_read2_sequence_trim.fastq 4000 91
-p MP4_4kb_read1_sequence_trim.fastq MP4_4kb_read2_sequence_trim.fastq 4000 136
-p MP5_4kb_read1_sequence_trim.fastq MP5_4kb_read2_sequence_trim.fastq 4000 138
-p MP6_4kb_read1_sequence_trim.fastq MP6_4kb_read2_sequence_trim.fastq 4000 133
-p MP7_4kb_read1_sequence_trim.fastq MP7_4kb_read2_sequence_trim.fastq 4000 133
-p MP8_4kb_read1_sequence_trim.fastq MP8_4kb_read2_sequence_trim.fastq 4000 133
-p MP1_10kb_read1_sequence_trim.fastq MP1_10kb_read2_sequence_trim.fastq 10000
99
-p MP2_10kb_read1_sequence_trim.fastq MP2_10kb_read2_sequence_trim.fastq 10000
97
-p MP3_10kb_read1_sequence_trim.fastq MP3_10kb_read2_sequence_trim.fastq 10000
98
-p MP4_10kb_read1_sequence_trim.fastq MP4_10kb_read2_sequence_trim.fastq 10000
98
-p MP5_10kb_read1_sequence_trim.fastq MP5_10kb_read2_sequence_trim.fastq 10000
98
-p MP6_10kb_read1_sequence_trim.fastq MP6_10kb_read2_sequence_trim.fastq 10000
98
-p MP7_10kb_read1_sequence_trim.fastq MP7_10kb_read2_sequence_trim.fastq 10000
97
-p MP8_10kb_read1_sequence_trim.fastq MP8_10kb_read2_sequence_trim.fastq 10000
97
-p Female-P1_85-90perc_trim.fastq Female-P2_85-90perc_trim.fastq 12500 2500
-p Male-P1_109-90perc_trim.fastq Male-P2_109-90perc_trim.fastq 6000 2000
-p Male-P1_86-90perc_trim.fastq Male-P2_86-90perc_trim.fastq 6000 2000
-p SP6_trim_Match.fastq T7_trim_Match.fastq 150000 1500
-s s_1_sequence_trim.fasta
-s s_2_sequence_trim.fasta
-s s_3_sequence_trim.fasta
-s s_4_sequence_trim.fasta
-s s_6_sequence_trim.fasta
-s s_7_sequence_trim.fasta
-s s_8_sequence_trim.fasta
-s PacSet1_F1_trim.fasta
-s PacSet1_F2_trim.fasta
-s PacSet1_F3_trim.fasta
-s PacSet1_F4_trim.fasta
-s F1_s1s2ccs_trim.fasta
-s F2_s1s2ccs_trim.fasta
-s F3_s1s2ccs_trim.fasta
-s F4_s1s2ccs_trim.fasta
-s F32JLNA01_trim.fasta
-s F32JLNA02_trim.fasta
-s F32JLNA03_trim.fasta
-s F32JLNA04_trim.fasta
-s F3EG98R01_trim.fasta
-s F3EG98R02_trim.fasta
-s F5V4F3A01_trim.fasta
-s F5V4F3A02_trim.fasta
-s F5ZTILC01_trim.fasta
-s F7JDG4X01_trim.fasta
-s F7JDG4X02_trim.fasta
-s All-Unigene.fa
-s Result_Novel_Brain.fasta
-s Result_Novel_Skin.fasta
-o /vlsci/VR0002/shared/canetoad/trimfiles/RayOutK31_barcooov2
-disable-recycling
-read-write-checkpoints BarcooRaycheck31
-minimum-contig-length 100
-enable-neighbourhoods
-write-read-markers
-write-extensions
-write-contig-paths
-write-marker-summary
-show-memory-usage
-show-memory-allocations
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
_______________________________________________
Denovoassembler-users mailing list
Denovoassembler-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
