Hi Gregg, Changing the Session.remove() to an expunge_all doesn't have any effect, the growth continues.
Chris On Apr 8, 4:10 pm, Gregg Lind <[email protected]> wrote: > I believe you want session.expunge() or session.expunge_all(). > > GL > > On Wed, Apr 8, 2009 at 6:05 PM, Chris Lewis <[email protected]> wrote: > > > > > > > Hi everyone! > > I have a threaded application which deals with a lot of records (it > > generates hundreds of thousands in an hour). I have a Database module > > that my program imports, and use scoped_session to make the session > > thread-safe. When a record is to be inserted, my program passes the > > object to be saved to the Database insert() function. Here's the code: > > > ============== > > > engine = sa.create_engine(engine_url, echo=echo) > > Session = sa.orm.scoped_session(sa.orm.sessionmaker(bind=engine, > > autocommit=False)) > > > def insert(obj): > > try: > > session().merge(obj) > > except Exception, e: > > log.warning("Database problem: " + str(e)) > > session().rollback() > > raise > > else: > > log.debug("Saved to database") > > > session().commit() > > Session.remove() > > > def session(): > > return Session() > > > ============== > > > Even though I call Session.remove(), it seems that I can't stop > > sqlalchemy.orm.identity.IdentityManagedState growing. Unit testing > > which inserts a couple thousand records shows the growth with Heapy. > > The "dict of sqlalchemy.orm.identity.IdentityManagedState" starts at > > 334 objects, ending with 11210 objects. > > > I thought Session.remove() would cause SQLAlchemy to release those > > resources, but this doesn't seem to be the case. As the process is > > going to need to long-running (weeks hopefully), I'm far happier with > > performing expensive CPU operations than exhausting my memory. > > > I am certain this is my own error, but I am not sure what it is. Any > > help would be appreciated! > > > Thanks in advance, > > Chris Lewis --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
