Hi Chris,
Thanks so much for the update. I did have a chance grab, compile, and run
the code. However, on my tests, I came across the same problem. However,
now we've reach an unexpected number of open files, namely, 2,097,148. I've
set both of the maxFileOpen/2 parameters to 4096 each. Are there supposed
to be that many number of open files?
The error message is below:
INFO buildNLMSA.main: Processing MAF file:
/nonsvn/data/UCSC/genomes/MOUSE/mm10/maf/chr17.maf
INFO:pygr-log:Processing MAF file:
/nonsvn/data/UCSC/genomes/MOUSE/mm10/maf/chr17.maf
[Traceback (most recent call last):
File
"/home/prigor/codebase/genomics/trunk/code/python/scripts/buildNLMSA.py",
line 93, in <module>
main()
File
"/home/prigor/codebase/genomics/trunk/code/python/scripts/buildNLMSA.py",
line 79, in main
maxOpenFiles2=4096)
File "cnestedlist.pyx", line 1508, in pygr.cnestedlist.NLMSA.__init__
File "cnestedlist.pyx", line 1794, in pygr.cnestedlist.NLMSA.readMAFfiles
File "cnestedlist.pyx", line 1622, in pygr.cnestedlist.NLMSA.newSequence
File "cnestedlist.pyx", line 1326, in
pygr.cnestedlist.NLMSASequence.__init__
IOError: unable to open in write mode:
/nonsvn/pygrdata/alignments/MOUSE/mm10/multiz60way/2097148.build
All the best,
Paul
--
Paul Rigor
http://www.ics.uci.edu/~prigor
On Tue, May 7, 2013 at 3:40 PM, Christopher Lee <cjlee...@gmail.com> wrote:
> Hi Paul,
> I implemented the method I described for limiting the maximum number of
> files open at any time. I have only performed very limited testing (on
> sacCer3 multiz7way MAF files), which simply showed that using this feature
> (by passing low value maxOpenFiles2=5 to NLMSA constructor) yielded
> identical index files vs. the standard mode not applying this limit.
>
> Maybe you could test this on your data. You don't need to change anything
> in the parameters you pass to NLMSA construction, since its new default
> values should work OK for you:
>
> * maxOpenFiles=1024 default is used only for requesting the OS to permit
> us to have up to 1024 open files;
> * maxOpenFiles2=1000 default will limit readMAFfiles() to 1000 open files
> at any one time, which should work for you.
>
> The code is available from my rebuf branch of my pygr repo on github:
> https://github.com/cjlee112/pygr/tree/rebuf
>
> Let me know whatever problems you run into...
>
> Yours,
>
> Chris
>
>
> On Tuesday, April 30, 2013 12:38:05 PM UTC-7, Paul Rigor wrote:
>>
>> Hi Chris, thanks so much for taking care of this. Looking forward to
>> testing the feature. Cheers,
>> Paul
>>
> --
> You received this message because you are subscribed to the Google Groups
> "pygr-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pygr-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to pygr-dev@googlegroups.com.
> Visit this group at http://groups.google.com/group/pygr-dev?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"pygr-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to pygr-dev+unsubscr...@googlegroups.com.
To post to this group, send email to pygr-dev@googlegroups.com.
Visit this group at http://groups.google.com/group/pygr-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
