Hi Chris, So I got in touch with the Pyrex developer and he re-posted version 0.9.8.6 (among other versions). But I still don't get a clean compilation even with the same version of pygr you use. In fact, I would have to rename __new__ to __cinit__ because otherwise the cnestedlist.pyx isn't parsed (ig, the c code is empty). Which version of GCC do you use? Thanks again! Paul
-- Paul Rigor http://www.ics.uci.edu/~prigor On Tue, Jun 18, 2013 at 9:23 AM, Paul Rigor <paul.ri...@uci.edu> wrote: > Hi Chris, > Sorry for the late reply, it was finals week last week. > Would you mind emailing me the source for Pyrex 0.9.8.6? It's not > available from the pyrex site's old release archive. > > When compiling using 0.9.9, 0.9.8.2, 0.9.8 -- there are tons more warning > messages than what you've listed. I do hope it's just an issue with the > version of pyrex. > > By the way, I misspoke regarding the 'number of intervals', I believe I > was referring to both 'maxint' and 'maxlen' where we wanted to decrease the > filesize of lpo db files because i/o performance becomes bottleneck for our > application. We'd prefer load several small lpo files (in the order KB) > rather than a giant one (in the order of GB) across thousands of separate > processes. > > Thanks again, > Paul > > > -- > Paul Rigor > http://www.ics.uci.edu/~prigor > > > On Thu, Jun 6, 2013 at 6:32 PM, Christopher Lee <cjlee...@gmail.com>wrote: > >> Not sure what the problem with your source code is. Pyrexc 0.9.8.6 runs >> fine on my latest cnestedlist.pyx, just prints a few warnings: >> >> (vehome)[user@work pygr]$ pyrexc --version >> Pyrex version 0.9.8.6 >> (vehome)[user@work pygr]$ pyrexc cnestedlist.pyx >> /home/user/projects/pygr/pygr/cnestedlist.pyx:8:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:51:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:167:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:371:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:424:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:446:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:1112:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> /home/user/projects/pygr/pygr/cnestedlist.pyx:1138:2: Warning: __new__ >> method of extension type will change semantics in a future version of >> Pyrex. Use __cinit__ instead. >> (vehome)[user@work pygr]$ ls -l cnestedlist.* >> -rw-r--r-- 1 user user 936552 Jun 6 18:26 cnestedlist.c >> -rw-r--r-- 1 user user 7452 May 7 23:02 cnestedlist.pxd >> -rw-r--r-- 1 user user 97515 May 7 23:09 cnestedlist.pyx >> >> >> BTW I don't know what you mean by "we'll definitely need to adjust the >> number of intervals when building the NLMSA"... >> >> >> On Thu, Jun 6, 2013 at 4:10 PM, Paul Rigor <paul.ri...@uci.edu> wrote: >> >>> Hi Chris, >>> The sample command above was the last of a series of slices we >>> attempted. But we'll definitely need to adjust the number of intervals when >>> building the NLMSA. The default settings generates huge files that take a >>> little bit too long to load. >>> >>> For the AttributeError, I've actually tried several versions of pyrex >>> (0.9.8, 0.9.8.2, 0.9.9), but I still can't re-compile the pyx files to c. >>> For all versions of pyrex, we get the following error when attempting to >>> compile cndestedlist.pyx and cdict.pyx >>> >>> pyrexc cnestedlist.pyx >>> /home/prigor/projects/msabuilder2/pygr/cnestedlist.pyx:7:55: Expected ')' >>> >>> pyrexc cdict.pyx >>> /home/prigor/projects/msabuilder2/pygr/cdict.pyx:129:23: Syntax error in >>> C variable declaration >>> >>> Which version of pyrex do you recommend we use? >>> Thanks! >>> Paul >>> >>> -- >>> Paul Rigor >>> http://www.ics.uci.edu/~prigor >>> >>> >>> On Wed, Jun 5, 2013 at 4:37 PM, Christopher Lee <cjlee...@gmail.com>wrote: >>> >>>> You are querying with the entire Y chromosome in your example, 91 MB in >>>> size, vs. a 60 genome alignment, is going to be a gigantic amount of >>>> aligned intervals. That may cause serious problems. I think you should >>>> try breaking that up into shorter intervals. E.g. start by trying with a 1 >>>> kb interval query, and then try longer queries if you need to. You should >>>> get the same results (querying 1 kb at a time) as you would get querying >>>> the whole thing... >>>> >>>> This error message is a weird traceback, which appears to show that >>>> Pyrex try-except handling is not working right. The doSlice call is inside >>>> a try... except AttributeError clause, so it should be impossible for it to >>>> raise an AttributeError! Can you try re-compiling from source, i.e. >>>> deleted cnestedlist.c, rerun the build (should remake cnestedlist.c using >>>> pyrexc)? Something seems to be wrong with your build. >>>> >>>> -- Chris >>>> >>>> >>>> On Wednesday, June 5, 2013 10:52:25 AM UTC-7, Vishal Patel wrote: >>>>> >>>>> Hi Chris, >>>>> >>>>> We are still having trouble with the MSA. We were able to build it >>>>> without any errors using pygr/0.8.2 >>>>> >>>>> However when we try to load and query it, we get the following error. >>>>> >>>>> In [1]: from pygr import worldbase >>>>> >>>>> In [2]: genome = worldbase.Bio.Seq.Genome.**MOUSE.mm10() >>>>> >>>>> In [3]: msa = worldbase.Bio.Seq.Alignments.**MOUSE.mm10.multiz60way() >>>>> >>>>> In [4]: slice = genome['chrY'] >>>>> >>>>> In [5]: res = msa[slice] >>>>> ------------------------------**------------------------------** >>>>> --------------- >>>>> AttributeError Traceback (most recent call >>>>> last) >>>>> <ipython-input-5-628143efd160> in <module>() >>>>> ----> 1 res = msa[slice] >>>>> >>>>> pygr/msa-builder/build/lib.**linux-x86_64-2.6/pygr/**cnestedlist.so >>>>> in pygr.cnestedlist.NLMSA.__**getitem__() >>>>> >>>>> pygr/msa-builder/build/lib.**linux-x86_64-2.6/pygr/**cnestedlist.so >>>>> in pygr.cnestedlist.NLMSASlice.__**cinit__() >>>>> >>>>> AttributeError: 'pygr.cnestedlist.NLMSA' object has no attribute >>>>> 'doSlice' >>>>> >>>>> Any clue? >>>>> >>>>> Vishal >>>>> >>>>> >>>>> >>>>> On Thursday, April 25, 2013 10:23:33 AM UTC-7, Vishal Patel wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> We usually download the pygr NLMSA from the UCLA server. However, the >>>>>> latest mm10 - multiz60way alignment does not exist there so we decided to >>>>>> build it. >>>>>> >>>>>> First I ran into issues with max size, >>>>>> >>>>>> * 1 msa = cnestedlist.NLMSA(pathstem=pathstem,* >>>>>> * 2 seqDict=genomeUnion,* >>>>>> *----> 3 mafFiles=maflist, mode="w")* >>>>>> * >>>>>> * >>>>>> *site-packages/pygr-0.8.2-py2.7-linux-x86_64.egg/pygr/cnestedlist.so >>>>>> in pygr.cnestedlist.NLMSA.__init__()* >>>>>> *site-packages/pygr-0.8.2-py2.7-linux-x86_64.egg/pygr/cnestedlist.so >>>>>> in pygr.cnestedlist.NLMSA.readMAFfiles() >>>>>> * >>>>>> * >>>>>> * >>>>>> *ValueError: MAF block too long! Increase max size* >>>>>> >>>>>> Setting the maxlen and maxint to a billion also did not help so I set >>>>>> it to sys.maxint. >>>>>> >>>>>> However this code still fails because it open 1000's of ".build" >>>>>> files and fails because it does not have "write permission" on the next >>>>>> file handle it tries to open. Basically hits the max open files limit. I >>>>>> increased the ulimit to 10,000 files at which point it fails with >>>>>> "Segmentation fault (core dumped)" >>>>>> >>>>>> Here is lsof out every 0.1 s when it failed, >>>>>> 170 # these are file handles to genome files >>>>>> 170 >>>>>> 170 >>>>>> 170 >>>>>> 170 >>>>>> 170 >>>>>> 170 >>>>>> 726 # starts creating .build files >>>>>> 1682 >>>>>> 2443 >>>>>> 3706 >>>>>> 6300 >>>>>> 7209 >>>>>> 9160 # hits the upper limit again and fails with a core dump. >>>>>> >>>>>> I have tried this with py2.6/pygr0.8.1 and py2.7/pygr0.8.2 and get >>>>>> the same error. >>>>>> >>>>>> Has anyone experienced this? >>>>>> What are the .build files? >>>>>> >>>>>> Vishal >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "pygr-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to pygr-dev+unsubscr...@googlegroups.com. >>>> To post to this group, send email to pygr-dev@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/pygr-dev?hl=en. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "pygr-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to pygr-dev+unsubscr...@googlegroups.com. >>> To post to this group, send email to pygr-dev@googlegroups.com. >>> Visit this group at http://groups.google.com/group/pygr-dev?hl=en. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "pygr-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to pygr-dev+unsubscr...@googlegroups.com. >> To post to this group, send email to pygr-dev@googlegroups.com. >> Visit this group at http://groups.google.com/group/pygr-dev?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- You received this message because you are subscribed to the Google Groups "pygr-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to pygr-dev+unsubscr...@googlegroups.com. To post to this group, send email to pygr-dev@googlegroups.com. Visit this group at http://groups.google.com/group/pygr-dev. For more options, visit https://groups.google.com/groups/opt_out.