thanks for the instant reply guys! as my app is on production so i cannot afford to bring things down right away for 0.4/0.5 migration. eventually, i will be going to (in next month) use 0.4/0.5. so for the time being (at least for the next one month) i am looking for the best solution on 0.3.x so that users are not affected.
michael, as you mentioned about explicit cleaning of session, i am doing that currently. let me quickly mention the flow of request so that you guys can have more information: - search request comes - if orm mapping is not created it's get created now (only happens one time) - new session is created and attached to the current thread (this is done so that different DAOs can access the same session from the current thread) - all orm queries are fired.. results processed - finally, current thread is accessed again, session attached earlier is accessed, session.clear() invoked and del session done. what's the best way to deal with the problem now... thanks, - A On Wed, Jun 18, 2008 at 7:49 PM, Michael Bayer <[EMAIL PROTECTED]> wrote: > > > On Jun 18, 2008, at 9:59 AM, Arun Kumar PG wrote: > > > hi all folks, > > > > i have a search form that allows user to search for records. i am > > eager loading 4 attributes on the master object which results in 4 > > left outer joins in the sa's sql query. the problem is that when i > > look at the memory consumption using top command it looks crazy. > > > > the memory shoots up by 50-60 MB instantly (some times even 100+ > > MB). i executed the query on db directly and the results are > > returned in 3 secs (close to around 60,000 rows). sa is spending a > > good amount of time processing the results and while it is doing > > that i see abnormal memory growth. also the cpu is used almost 98% > > during this time. > > > > the interesting thing is that after processing the request the > > memory does not comes down. it stays there only. i dont know why its > > not gc'ed. > > > > my environment: > > - mysql 4.1 > > - sa 3.9 > > - python 2.4 > > > > is there any chance that memory is getting leaked as i don't see > > memory come down even after some time. > > > > The Session in 0.3 does not lose references to any data loaded > automatically, it has to be cleaned out manually using > session.expunge(obj) or session.clear(). From 0.4 on forward the > Session is weak referencing so that unreferenced, clean objects fall > out of scope automatically. 0.4 also eager loads many rows about 30% > faster than 0.3 and 0.5 is then about 15% faster than 0.4. ORMs in > general are designed for rich in-memory functionality and are not > optimized for loads of many tens of thousands of rows, so for better > performance overall consider non-ORM access to these rows. > > > > -- cheers, - a --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
