Hi Michael
I'm having an issue with memory usage that I would appreciate some insight..
I have a fairly straight forward process, that works perfectly as far as it
delivering the desired updates in the DB etc, however, it accumulates memory
usage (just like a leak) and I can not find a way to release it.
I have a multi threaded process. Each thread creates a scopedsession from the
standard global Session = scoped_session(sessionmaker()) construct. Each
thread does some work using mapped objects to update some and add some rows
into the (MySQL on Windows) DB. All this works perfectly and as expected.
Due to various reasons, I flush/commit after each row is updated/inserted.
After the "batch" of updates is complete, I come back and session.remove() (In
an attempt to dump the session) and then wait for a while and do the entire
thing again. At the start of each run, I create a new session=Session() and do
the updates and return and session.remove().
To me, I would assume that the memory would be the session's cache of objects
that are being managed - which I can understand. What I can't understand is
why when I delete *everything* e.g. del engine, del meta, del session and even
stop the thread, the memory is still consumed. I must stop the entire
process before the memory is returned to the system. After around 10 hours of
running, I've used 2Gb+ of memory and everything crashes.
BTW: I have created a version of my code that does everything EXCEPT the SA
part(s), and no memory is being used at all. (Just checking that it wasn't my
own code causing the issue!) i.e. It loops over the other database (non SQL)
reading all the data that I WOULD use to update the SQL database using SA.
When SA is not involved, nothing is happening with the memory.
Any hint on how I can a) see what is being held and b) dump it!
I'm using 0.6.3 and Python 2.6.3 on Windows. BTW: I tried to update to latest
7.x and Python 2.7.2 however that broke everything in a spectacular way - I'll
leave that one for another day..
Cheers
Warwick
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
