OK, I tried running the tests you outlined, but couldn't recreate the error with the development version of Ray you provided with or without recycling. When I went back to the non-development version of 2.0.0rc8, the error occurred again (3rd time). I noticed that the development platform version was 1.0.3, but the platform version on my error-giving software was 1.0.2. Did anything else change (other than the -disable-recycling switch) between those versions that might explain the difference?
See the run summaries below: Run1: (recycling disabled) Ray version 2.0.0-rc8+devel License for Ray: GNU General Public License version 3 RayPlatform version: 1.0.3devel License for RayPlatform: GNU Lesser General Public License version 3 -disable-recycling \ -p \ ../all_reads/100x_reads/BL077_100x_1.fastq.gz \ ../all_reads/100x_reads/BL077_100x_2.fastq.gz \ -o \ BL077_noRC \ -read-write-checkpoints \ BL077.Checkpoints \ -k \ 31 \ Result: Finished assembly. --------------------------------------------------------------------------- Run 2: (recycling not disabled, but using the same checkpoints as Run 1, so seed extension was skipped) Ray version 2.0.0-rc8+devel License for Ray: GNU General Public License version 3 RayPlatform version: 1.0.3devel License for RayPlatform: GNU Lesser General Public License version 3 -disable-recycling.NO \ -p \ ../all_reads/100x_reads/BL077_100x_1.fastq.gz \ ../all_reads/100x_reads/BL077_100x_2.fastq.gz \ -o \ BL077_yesRC \ -read-write-checkpoints \ BL077.Checkpoints \ -k \ 31 \ Result: Finished assembly. ----------------------------------------------------------------------- Run3: (recycling not disabled, but now writing its own checkpoint file instead of using prior checkpoints) Ray version 2.0.0-rc8+devel License for Ray: GNU General Public License version 3 RayPlatform version: 1.0.3devel License for RayPlatform: GNU Lesser General Public License version 3 -disable-recycling.NO \ -p \ ../all_reads/100x_reads/BL077_100x_1.fastq.gz \ ../all_reads/100x_reads/BL077_100x_2.fastq.gz \ -o \ BL077_yesRC2 \ -read-write-checkpoints \ BL077.Checkpoints2 \ -k \ 31 \ Result: Finished assembly -------------------------------------------------------------------------- Run4: (recycling disable switch omitted, which I assume allows recycling by default, and no checkpointing) Ray version 2.0.0-rc8+devel License for Ray: GNU General Public License version 3 RayPlatform version: 1.0.3devel License for RayPlatform: GNU Lesser General Public License version 3 -p \ ../all_reads/100x_reads/BL077_100x_1.fastq.gz \ ../all_reads/100x_reads/BL077_100x_2.fastq.gz \ -k \ 31 \ -o \ BL077_yesRC3 \ Result: Finished assembly -------------------------------------------------------------------------- Run5: (using the non-development version with the same commands as Run4) Ray version 2.0.0-rc8 License for Ray: GNU General Public License version 3 RayPlatform version: 1.0.2 License for RayPlatform: GNU Lesser General Public License version 3 -p \ ../all_reads/100x_reads/BL077_100x_1.fastq.gz \ ../all_reads/100x_reads/BL077_100x_2.fastq.gz \ -k \ 31 \ -o \ BL077_100x_oldRay \ Result: Infinite loop on seed 284. On Jun 15, 2012, at 10:03 AM, Sébastien Boisvert wrote: > Hi, > > Egon Ozer a écrit : >> The assembly completed with the -disable-recycling flag. I just tried >> re-running it with the -disable-recycling.NO, but because I used >> checkpointing, it seems to have skipped the entire seed extension step and >> thus completed again. I'm trying it again with -disable-recycling.NO and no >> checkpointing just to see if the error happens again with this development >> version you sent me. > > You can cherry-pick checkpoint files manually and delete them before > launching the job. > > In your case, you want to restart at the bidrectional extension. > > So you need to delete these checkpoints: > > rm TestX-112.CHECK/*.Extensions.ray TestX-112.CHECK/*ContigPaths.ray > > And re-run the command. > > > You command needs the same number of arguments that you used when you > generated > checkpoints. > >> Interestingly, there were less seeds with the -disable-recycling. As I >> wrote below, seed 285 was the one getting stuck before. On this most recent >> run, the last seed was 252. Is that expected? > > -disable-recycling only affects the biddirectional extension. > > Basically, it disables some algorithms that optimize read placement onto > contig paths. > > Apparently, it caused some strange stuff in your case. > >> - Egon >> >> >> On Jun 14, 2012, at 1:35 PM, Sébastien Boisvert wrote: >> >>> Hello, >>> >>> I just added the option -disable-recycling, to check if it is due to read >>> recycling. >>> >>> Can you install the last version available in git to test this ? >>> >>> >>> git clone git://github.com/sebhtml/ray.git >>> git clone git://github.com/sebhtml/RayPlatform.git >>> >>> cd ray >>> make PREFIX=$(pwd)/Build >>> make install >>> >>> >>> Also, to avoid recomputing the same initial stuff over and over again, you >>> can use >>> checkpointing. >>> >>> >>> mpiexec -n 32 \ >>> Ray -k 31 -o Test3 -p SampleX/file1.fastq SampleX/file2.fastq \ >>> -disable-recycling \ >>> -read-write-checkpoints SampleX.Checkpoints >>> >>> Then you can re-run it again >>> >>> mpiexec -n 32 \ >>> Ray -k 31 -o Test4 -p SampleX/file1.fastq SampleX/file2.fastq \ >>> -disable-recycling.NO \ >>> -read-write-checkpoints SampleX.Checkpoints >>> >>> >>> Egon Ozer a écrit : >>>> Just re-ran it the data set with the same settings and it got stuck in the >>>> same loop at the same place, it seems. See following: >>>> >>>> Rank 2 starts on seed 285, length is 93, flow 0 [285/291] >>>> Current peak coverage -> 164 >>>> Rank 2 reached 0 vertices from seed 285, flow 1 >>>> Speed RAY_SLAVE_MODE_EXTENSION 0 units/second >>>> Rank 2: assembler memory usage: 0 KiB >>>> ... >>>> Rank 2 reached 8270200 vertices from seed 285, flow 1 >>>> Speed RAY_SLAVE_MODE_EXTENSION 2195 units/second >>>> Rank 2: assembler memory usage: 0 KiB >>>> Rank 2 reached 8270300 vertices from seed 285, flow 1 >>>> Speed RAY_SLAVE_MODE_EXTENSION 2670 units/second >>>> Rank 2: assembler memory usage: 0 KiB >>>> Rank 2 reached 8270400 vertices from seed 285, flow 1 >>>> Speed RAY_SLAVE_MODE_EXTENSION 2545 units/second >>>> Rank 2: assembler memory usage: 0 KiB >>>> Rank 2 reached 8270500 vertices from seed 285, flow 1 >>>> Speed RAY_SLAVE_MODE_EXTENSION 2874 units/second >>>> Rank 2: assembler memory usage: 0 KiB >>>> Rank 2 reached 8270600 vertices from seed 285, flow 1 >>>> Speed RAY_SLAVE_MODE_EXTENSION 2742 units/second >>>> Rank 2: assembler memory usage: 0 KiB >>>> >>>> >>>> >>>> >>>> On Jun 14, 2012, at 10:27 AM, Sébastien Boisvert wrote: >>>> >>>>> Hello, >>>>> >>>>> >>>>>> Rank 2 reached 76881500 vertices from seed 285, flow 1 >>>>> As you said, there seems to be an infinite loop as 76881500 vertices is >>>>> very long. >>>>> >>>>> >>>>> I never saw this behavior before. >>>>> >>>>> >>>>> >>>>> There should be a line somewhere in your log like this: >>>>> >>>>> Rank 2 starts on seed 285, length is<seedLength>, flow 0 >>>>> Current peak coverage -> <localPeakCoverage> >>>>> >>>>> >>>>> What is the length of the seed 285 in your computation ? >>>>> >>>>> >>>>> >>>>> Is the problem reproducible with the same data and the same exact >>>>> command on the >>>>> same machine ? >>>>> >>>>> >>>>> Also, the reported memory usage is 0 kb because this feature is not >>>>> implemented >>>>> on Mac OS. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Sébastien >>>>> >>>>> Egon Ozer a écrit : >>>>>> Rank 2: assembler memory usage: 0 KiB >>>>> ------------------------------------------------------------------------------ >>>>> Live Security Virtual Conference >>>>> Exclusive live event will cover all the ways today's security and >>>>> threat landscape has changed and how IT managers can respond. Discussions >>>>> will include endpoint security, mobile security and the latest in malware >>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>>>> _______________________________________________ >>>>> Denovoassembler-users mailing list >>>>> Denovoassembler-users@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/denovoassembler-users > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Denovoassembler-users mailing list Denovoassembler-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
