Yup -- Our homegrown subclass of DBServerInfo cleans up after SQLAlchemy.
What other functionality does DBServerInfo provide aside from establishing a
db connection and providing cursor(s) to the db?
Thanks,
Paul
On Tue, Feb 23, 2010 at 3:16 PM, Kenny Daily <kmda...@gmail.com> wrote:
> Looks like the homegrown code is the problem. Using DBServerInfo as
> provided doesn't reproduce this memory usage. It may be that using
> sqlalchemy directly to generate a cursor is doing something with all
> of the tables in a database...
>
> Kenny
>
> On Feb 22, 5:49 pm, Christopher Lee <l...@chem.ucla.edu> wrote:
> > On Feb 22, 2010, at 5:37 PM, Kenny Daily wrote:
> >
> > > from pygr import worldbase
> > > all_resources = worldbase.dir("Bio.Test.Annotation")
> > > dbs = filter(lambda x: x.endswith("db"), all_resources)
> >
> > > for x in dbs:
> > > resource = worldbase(x)
> > > resource.close()
> >
> > > This uses ~50-100MB per resource loaded, and it accumulates. In my
> > > case, the AnnotationDB's sliceDB is a MySQL table. Any ideas about
> > > where the memory could be going?
> >
> > Our XMLRPC server loads all the AnnotationDB that it serves (must be at
> least 100 - 200) with little memory usage, so this horrible memory usage
> should absolutely not be happening to you. If you tested this with beta1
> code, please re-run your example with an official release i.e. 0.8.0 or
> 0.8.1. I suspect that this may solve your problem -- as I recall we
> discovered a nasty bug in MySQLdb that uses up massive amounts of memory
> when you simply ask for an iterator. Starting with 0.8.0 we included a
> workaround to avoid that problem.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pygr-dev" group.
> To post to this group, send email to pygr-...@googlegroups.com.
> To unsubscribe from this group, send email to
> pygr-dev+unsubscr...@googlegroups.com<pygr-dev%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/pygr-dev?hl=en.
>
>
--
Paul Rigor
Pre-doctoral BIT Fellow and Graduate Student
Institute for Genomics and Bioinformatics
Donald Bren School of Information and Computer Sciences
University of California, Irvine
http://www.ics.uci.edu/~prigor
--
You received this message because you are subscribed to the Google Groups
"pygr-dev" group.
To post to this group, send email to pygr-...@googlegroups.com.
To unsubscribe from this group, send email to
pygr-dev+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pygr-dev?hl=en.
