On 01/12/13 04:12 AM, Sucheta Tripathy wrote: > > I am trying to run Ray for a while now on our Illumina data. It is run in > parallel mode and memory is not a constraint now because we have a cluster > with 16 X 16 GB memory in each node. >
The problem is this: tmp1 and tmp2 don't contain the same number of sequences. Using paired files with different number of sequences will cause crash in Ray 2.2.0 and earlier versions. If you use Ray 2.3.0, Ray will throw an error. Also, lane1_1b and lane_1_2b are empty (0 sequences). You may want to check that out too. > my Command: > mpiexec -n 64 Ray \ > -k \ > 31 \ > -disable-recycling \ > -p \ > /scratch/sucheta/tmp1.fastq \ > /scratch/sucheta/tmp2.fastq \ > -p \ > /scratch/sucheta/lane1_1b.fq \ > /scratch/sucheta/lane1_2b.fq \ > -o \ > rayTest > > Content of NumberOfSequences.txt > Files: 4 > > FileNumber: 0 > FilePath: /scratch/sucheta/tmp1.fastq > NumberOfSequences: 73013670 > FirstSequence: 0 > LastSequence: 73013669 > > FileNumber: 1 > FilePath: /scratch/sucheta/tmp2.fastq > NumberOfSequences: 72339350 > FirstSequence: 73013670 > LastSequence: 145353019 > > FileNumber: 2 > FilePath: /scratch/sucheta/lane1_1b.fq > NumberOfSequences: 0 > > FileNumber: 3 > FilePath: /scratch/sucheta/lane1_2b.fq > NumberOfSequences: 0 > > Summary > NumberOfSequences: 145353020 > FirstSequence: 0 > LastSequence: 145353019 > > > However, after running for a while Ray terminates: This is the last few lines > of STDOUT: > > --- > > Speed RAY_SLAVE_MODE_EXTENSION 1183 units/second > Rank 9: assembler memory usage: 850432 KiB > Rank 9 is changing direction. > Rank 12: assembler memory usage: 826944 KiB > Rank 12 starts on seed 1200, length is 477, flow 0 [1200/6451] > Rank 12 starts on seed 1201, length is 476, flow 0 [1201/6451] > Rank 10 traversed 1273646 nucleotide symbols > Rank 10: assembler memory usage: 869460 KiB > Rank 10 starts on seed 534, length is 906, flow 0 [534/6579] > Rank 31 reached 2270 vertices from seed 491, flow 1 > Speed RAY_SLAVE_MODE_EXTENSION 1174 units/second > Rank 31: assembler memory usage: 866768 KiB > Rank 31 is changing direction. > Rank 2 traversed 1910058 nucleotide symbols > Rank 2 reached 1269 vertices from seed 1007, flow 2 > Speed RAY_SLAVE_MODE_EXTENSION 1435 units/second > Rank 2: assembler memory usage: 843432 KiB > Rank 2 (extension done) NumberOfFlows: 2 > Rank 2 FlowedVertices: 0 576 1 1269 2 1269 > Rank 2: assembler memory usage: 839332 KiB > Rank 2 starts on seed 1008, length is 576, flow 0 [1008/6537] > Terminated > --- > > It creates the following output files before crashing: > > CoverageDistributionAnalysis.txt NumberOfSequences.txt > CoverageDistribution.txt ParallelPaths.txt > degreeDistribution.txt Plugins > ElapsedTime.txt RayCommand.txt > FilePartition.txt RayPlatform_Version.txt > GraphPartition.txt RayVersion.txt > LibraryData.xml Scheduling > LibraryStatistics.txt SeedLengthDistribution.txt > NetworkTest.txt SequencePartition.txt > > > Any help in this regard will be much appreciated! > > > -- > Sucheta Tripathy, Ph.D > Scientist, Ramalingaswamy Fellow, > Indian Institute of Chemical Biology, > Kolkata, India. > https://sites.google.com/site/suchetalab/ > https://twitter.com/tsucheta ------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk _______________________________________________ Denovoassembler-users mailing list Denovoassembler-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
