Hi Chris,
So I got in touch with the Pyrex developer and he re-posted version 0.9.8.6
(among other versions). But I still don't get a clean compilation even with
the same version of pygr you use. In fact, I would have to rename __new__
to __cinit__ because otherwise the cnestedlist.pyx isn't parsed (ig, the c
code is empty). Which version of GCC do you use?
Thanks again!
Paul

--
Paul Rigor
http://www.ics.uci.edu/~prigor


On Tue, Jun 18, 2013 at 9:23 AM, Paul Rigor <paul.ri...@uci.edu> wrote:

> Hi Chris,
> Sorry for the late reply, it was finals week last week.
> Would you mind emailing me the source for Pyrex 0.9.8.6? It's not
> available from the pyrex site's old release archive.
>
> When compiling using 0.9.9, 0.9.8.2, 0.9.8 -- there are tons more warning
> messages than what you've listed. I do hope it's just an issue with the
> version of pyrex.
>
> By the way, I misspoke regarding the 'number of intervals', I believe I
> was referring to both 'maxint' and 'maxlen' where we wanted to decrease the
> filesize of lpo db files because i/o performance becomes bottleneck for our
> application. We'd prefer load several small lpo files (in the order KB)
> rather than a giant one (in the order of GB) across thousands of separate
> processes.
>
> Thanks again,
> Paul
>
>
> --
> Paul Rigor
> http://www.ics.uci.edu/~prigor
>
>
> On Thu, Jun 6, 2013 at 6:32 PM, Christopher Lee <cjlee...@gmail.com>wrote:
>
>> Not sure what the problem with your source code is.  Pyrexc 0.9.8.6 runs
>> fine on my latest cnestedlist.pyx, just prints a few warnings:
>>
>> (vehome)[user@work pygr]$ pyrexc --version
>> Pyrex version 0.9.8.6
>> (vehome)[user@work pygr]$ pyrexc cnestedlist.pyx
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:8:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:51:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:167:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:371:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:424:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:446:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:1112:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> /home/user/projects/pygr/pygr/cnestedlist.pyx:1138:2: Warning: __new__
>> method of extension type will change semantics in a future version of
>> Pyrex. Use __cinit__ instead.
>> (vehome)[user@work pygr]$ ls -l cnestedlist.*
>> -rw-r--r-- 1 user user  936552 Jun  6 18:26 cnestedlist.c
>> -rw-r--r-- 1 user user    7452 May  7 23:02 cnestedlist.pxd
>> -rw-r--r-- 1 user user   97515 May  7 23:09 cnestedlist.pyx
>>
>>
>> BTW I don't know what you mean by "we'll definitely need to adjust the
>> number of intervals when building the NLMSA"...
>>
>>
>> On Thu, Jun 6, 2013 at 4:10 PM, Paul Rigor <paul.ri...@uci.edu> wrote:
>>
>>> Hi Chris,
>>> The sample command above was the last of a series of slices we
>>> attempted. But we'll definitely need to adjust the number of intervals when
>>> building the NLMSA. The default settings generates huge files that take a
>>> little bit too long to load.
>>>
>>> For the AttributeError, I've actually tried several versions of pyrex
>>> (0.9.8, 0.9.8.2, 0.9.9), but I still can't re-compile the pyx files to c.
>>> For all versions of pyrex, we get the following error when attempting to
>>> compile cndestedlist.pyx and cdict.pyx
>>>
>>> pyrexc cnestedlist.pyx
>>> /home/prigor/projects/msabuilder2/pygr/cnestedlist.pyx:7:55: Expected ')'
>>>
>>> pyrexc cdict.pyx
>>> /home/prigor/projects/msabuilder2/pygr/cdict.pyx:129:23: Syntax error in
>>> C variable declaration
>>>
>>> Which version of pyrex do you recommend we use?
>>> Thanks!
>>> Paul
>>>
>>> --
>>> Paul Rigor
>>> http://www.ics.uci.edu/~prigor
>>>
>>>
>>> On Wed, Jun 5, 2013 at 4:37 PM, Christopher Lee <cjlee...@gmail.com>wrote:
>>>
>>>> You are querying with the entire Y chromosome in your example, 91 MB in
>>>> size, vs. a 60 genome alignment, is going to be a gigantic amount of
>>>> aligned intervals.  That may cause serious problems.  I think you should
>>>> try breaking that up into shorter intervals.  E.g. start by trying with a 1
>>>> kb interval query, and then try longer queries if you need to.  You should
>>>> get the same results (querying 1 kb at a time) as you would get querying
>>>> the whole thing...
>>>>
>>>> This error message is a weird traceback, which appears to show that
>>>> Pyrex try-except handling is not working right.  The doSlice call is inside
>>>> a try... except AttributeError clause, so it should be impossible for it to
>>>> raise an AttributeError!  Can you try re-compiling from source, i.e.
>>>> deleted cnestedlist.c, rerun the build (should remake cnestedlist.c using
>>>> pyrexc)?  Something seems to be wrong with your build.
>>>>
>>>> -- Chris
>>>>
>>>>
>>>> On Wednesday, June 5, 2013 10:52:25 AM UTC-7, Vishal Patel wrote:
>>>>>
>>>>> Hi Chris,
>>>>>
>>>>> We are still having trouble with the MSA. We were able to build it
>>>>> without any errors using pygr/0.8.2
>>>>>
>>>>> However when we try to load and query it, we get the following error.
>>>>>
>>>>> In [1]: from pygr import worldbase
>>>>>
>>>>> In [2]: genome = worldbase.Bio.Seq.Genome.**MOUSE.mm10()
>>>>>
>>>>> In [3]: msa = worldbase.Bio.Seq.Alignments.**MOUSE.mm10.multiz60way()
>>>>>
>>>>> In [4]: slice = genome['chrY']
>>>>>
>>>>> In [5]: res = msa[slice]
>>>>> ------------------------------**------------------------------**
>>>>> ---------------
>>>>> AttributeError                            Traceback (most recent call
>>>>> last)
>>>>> <ipython-input-5-628143efd160> in <module>()
>>>>> ----> 1 res = msa[slice]
>>>>>
>>>>> pygr/msa-builder/build/lib.**linux-x86_64-2.6/pygr/**cnestedlist.so
>>>>> in pygr.cnestedlist.NLMSA.__**getitem__()
>>>>>
>>>>> pygr/msa-builder/build/lib.**linux-x86_64-2.6/pygr/**cnestedlist.so
>>>>> in pygr.cnestedlist.NLMSASlice.__**cinit__()
>>>>>
>>>>> AttributeError: 'pygr.cnestedlist.NLMSA' object has no attribute
>>>>> 'doSlice'
>>>>>
>>>>> Any clue?
>>>>>
>>>>> Vishal
>>>>>
>>>>>
>>>>>
>>>>> On Thursday, April 25, 2013 10:23:33 AM UTC-7, Vishal Patel wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We usually download the pygr NLMSA from the UCLA server. However, the
>>>>>> latest mm10 - multiz60way alignment does not exist there so we decided to
>>>>>> build it.
>>>>>>
>>>>>> First I ran into issues with max size,
>>>>>>
>>>>>> *       1 msa = cnestedlist.NLMSA(pathstem=pathstem,*
>>>>>> *       2                             seqDict=genomeUnion,*
>>>>>> *----> 3                             mafFiles=maflist, mode="w")*
>>>>>> *
>>>>>> *
>>>>>> *site-packages/pygr-0.8.2-py2.7-linux-x86_64.egg/pygr/cnestedlist.so
>>>>>> in pygr.cnestedlist.NLMSA.__init__()*
>>>>>> *site-packages/pygr-0.8.2-py2.7-linux-x86_64.egg/pygr/cnestedlist.so
>>>>>> in pygr.cnestedlist.NLMSA.readMAFfiles()
>>>>>> *
>>>>>> *
>>>>>> *
>>>>>> *ValueError: MAF block too long!  Increase max size*
>>>>>>
>>>>>> Setting the maxlen and maxint to a billion also did not help so I set
>>>>>> it to sys.maxint.
>>>>>>
>>>>>> However this code still fails because it open 1000's of ".build"
>>>>>> files and fails because it does not have "write permission" on the next
>>>>>> file handle it tries to open. Basically hits the max open files limit. I
>>>>>> increased the ulimit to 10,000 files at which point it fails with
>>>>>> "Segmentation fault (core dumped)"
>>>>>>
>>>>>> Here is lsof out every 0.1 s when it failed,
>>>>>> 170 # these are file handles to genome files
>>>>>> 170
>>>>>> 170
>>>>>> 170
>>>>>> 170
>>>>>> 170
>>>>>> 170
>>>>>> 726 # starts creating .build files
>>>>>> 1682
>>>>>> 2443
>>>>>> 3706
>>>>>> 6300
>>>>>> 7209
>>>>>> 9160 # hits the upper limit again and fails with a core dump.
>>>>>>
>>>>>> I have tried this with py2.6/pygr0.8.1 and py2.7/pygr0.8.2 and get
>>>>>> the same error.
>>>>>>
>>>>>> Has anyone experienced this?
>>>>>> What are the .build files?
>>>>>>
>>>>>> Vishal
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "pygr-dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to pygr-dev+unsubscr...@googlegroups.com.
>>>> To post to this group, send email to pygr-dev@googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/pygr-dev?hl=en.
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>>
>>>>
>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "pygr-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to pygr-dev+unsubscr...@googlegroups.com.
>>> To post to this group, send email to pygr-dev@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/pygr-dev?hl=en.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "pygr-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pygr-dev+unsubscr...@googlegroups.com.
>> To post to this group, send email to pygr-dev@googlegroups.com.
>> Visit this group at http://groups.google.com/group/pygr-dev?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pygr-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pygr-dev+unsubscr...@googlegroups.com.
To post to this group, send email to pygr-dev@googlegroups.com.
Visit this group at http://groups.google.com/group/pygr-dev.
For more options, visit https://groups.google.com/groups/opt_out.


Reply via email to