I used weak ref dict and doing a session.clear() post every request. But when I do a len(gc.get_objects()) post every request I can see that the number of objects keeps on increasing. Do we have a background thread that frees up the unused objects in SA after a while?
On Dec 7, 2007 3:49 AM, Michael Bayer <[EMAIL PROTECTED]> wrote: > > > On Dec 6, 2007, at 4:58 PM, Felix Schwarz wrote: > > > > > Michael Bayer schrieb: > >> in version 0.4, the session is weak referencing so that objects > >> which are not elsewhere referenced (and also are not marked as > >> dirty or deleted) fall out of scope automatically. that is > >> documented at: > > > > http://www.sqlalchemy.org/docs/04/session.html#unitofwork_using_attributes > > > > I have a question which I think is similar enough to be asked in the > > same thread: I have a set of quite simple migration scripts which us > > SQLAlchemy 0.4 and Elixir 0.4. I do extract data from the old legacy > > (MySQL) database with SQLAlchemy and put this data into new Elixir > > objects. > > > > Currently, these scripts use up to 600 MB RAM. This is no real > > problem as we probably could devote a machine with 4 GB ram solely > > for the automated migration. But it would be nice to use lower- > > powered machines for our migration tasks. > > > > What wonders me is that I do not (knowingly) keep references neither > > to the old data items nor the new elixir objects. Nevertheless > > memory usage increases during the migration. Is there any way to > > debug this easily to see why Python does need so much memory/which > > references prevent the objects from being garbage collected? Running > > the garbage collector manually did not help much (saving only about > > 5 MB). > > > > I usually dig into gc.get_objects() to try to see whats up. For your > migration script, what happens if you issue session.flush() > periodically ? > > > > > > -- Cheers, - A --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
