On 23/09/13 12:37 AM, Ranjeev wrote: > I managed to get Ray running however, I get the some memory problem which I > cant really understand. I ran the pbs on three nodes without multiple > processes and still the error shows up.
Hello, It seems to me that 145745448 reads is a lot of reads for a single MPI rank. However, "-80285528 bytes" is definitely a bug in Ray. Which version of Ray are you using ? > > qsub -l nodes=compute-0-0+compute-0-1+compute-0-2 ray_p.pbs > > Attached is the log. Below the summary > > *** > Step: Sequence loading > Date: Mon Sep 23 10:50:19 2013 > Elapsed time: 15 minutes, 57 seconds > Since beginning: 27 minutes, 2 seconds > *** > > > Rank 2 has 145745448 sequence reads (completed) > Critical exception: The system is out of memory, returned NULL. > Requested -80285528 bytes of type RAY_MALLOC_TYPE_BLOOM_FILTER > Critical exception: The system is out of memory, returned NULL. > Requested -80285528 bytes of type RAY_MALLOC_TYPE_BLOOM_FILTER > Critical exception: The system is out of memory, returned NULL. > Requested -80285528 bytes of type RAY_MALLOC_TYPE_BLOOM_FILTER > -------------------------------------------------------------------------- > mpiexec has exited due to process rank 0 with PID 8831 on > node compute-0-0.local exiting improperly. There are two reasons this could > occur: > > 1. this process did not call "init" before exiting, but others in > the job did. This can cause a job to hang indefinitely while it waits > for all processes to call "init". By rule, if one process calls "init", > then ALL processes must call "init" prior to termination. > > 2. this process called "init", but exited without calling "finalize". > By rule, all processes that call "init" MUST call "finalize" prior to > exiting or it will be considered an "abnormal termination" > > This may have caused other processes in the application to be > terminated by signals sent by mpiexec (as reported here). > -------------------------------------------------------------------------- > > *Hope you can help* > > > Thanks. > > > > > On Sun, Sep 22, 2013 at 2:28 PM, Ranjeev <ranj...@um.edu.my > <mailto:ranj...@um.edu.my>> wrote: > > Btw, my assembly of the panda genome terminated with PBS: job killed: > node 2 (compute-0-4) requested job terminate, 'EOF' (code 1099) - received > SISTER_EOF attempting to communicate with sister MOM's. > > > On Sun, Sep 22, 2013 at 12:21 PM, Ranjeev <ranj...@um.edu.my > <mailto:ranj...@um.edu.my>> wrote: > > Hi > > Thanks I was using torque and specified the nodes only with > Infiniband and it seemed to have added those n parameters by itself. Btw, if > I have a node with 192Gb RAM and 24 processors, how many processes do you > recommend per node? I notice each process consuming considerable RAM and > start to use swap when I use 24 processes per node. > > > > On Fri, Sep 20, 2013 at 7:08 PM, Sébastien Boisvert > <sebastien.boisver...@ulaval.ca <mailto:sebastien.boisver...@ulaval.ca>> > wrote: > > On 19/09/13 11:26 PM, Ranjeev wrote: > > HI, > > I'm using openmpi and torque - I managed to test Ray once and > worked fine (for a single node). > The next time I run the network test takes forever. Doesn't > go beyond the last line > > Ray command: > mpiexec -n 2 Ray \ > -n \ > 40 \ > > > > Do you want 2 ranks or 40 ranks ? > > > > -p \ > /home/ranjeev/NTM/L007_R1.fq \ > /home/ranjeev/NTM/L007_R2.fq \ > -o \ > /home/ranjeev/temp/Ray/test3 > > Rank 0 wrote /home/ranjeev/temp/Ray/test3/__RayCommand.txt > > k-mer length: 21 > Rank 1: assembler memory usage: 24972 KiB > Rank 0: assembler memory usage: 24976 KiB > Rank 1: assembler memory usage: 90796 KiB > Rank 1: Rank= 1 Size= 2 ProcessIdentifier= 12290 > Rank 0: assembler memory usage: 90804 KiB > Rank 0: Rank= 0 Size= 2 ProcessIdentifier= 18150 > Rank 0: testing the network, please wait... > > Rank 0 is testing the network [0/1000] > > > Regards, > Ranjeev > > > > -- > " PENAFIAN: E-mel ini dan apa-apa fail yang dikepilkan bersamanya > ("Mesej") adalah ditujukan hanya untuk kegunaan penerima(-penerima) yang > termaklum di atas dan mungkin mengandungi maklumat sulit. Anda dengan ini > dimaklumkan bahawa mengambil apa jua tindakan bersandarkan kepada, membuat > penilaian, mengulang hantar, menghebah, mengedar, mencetak, atau menyalin > Mesej ini atau sebahagian daripadanya oleh sesiapa selain daripada > penerima(-penerima) yang termaklum di atas adalah dilarang. Jika anda telah > menerima Mesej ini kerana kesilapan, anda mesti menghapuskan Mesej ini dengan > segera dan memaklumkan kepada penghantar Mesej ini menerusi balasan e-mel. > Pendapat-pendapat, rumusan-rumusan, dan sebarang maklumat lain di dalam Mesej > ini yang tidak berkait dengan urusan rasmi Universiti Malaya adalah difahami > sebagai bukan dikeluar atau diperakui oleh mana-mana pihak yang disebut. > > > DISCLAIMER: This e-mail and any files transmitted with it > ("Message") is intended only for the use of the recipient(s) named above and > may contain confidential information. You are hereby notified that the taking > of any action in reliance upon, or any review, retransmission, dissemination, > distribution, printing or copying of this Message or any part thereof by > anyone other than the intended recipient(s) is strictly prohibited. If you > have received this Message in error, you should delete this Message > immediately and advise the sender by return e-mail. Opinions, conclusions and > other information in this Message that do not relate to the official business > of University of Malaya shall be understood as neither given nor endorsed by > any of the forementioned. " > > > > ------------------------------------------------------------------------------ LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk _______________________________________________ Denovoassembler-users mailing list Denovoassembler-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
