actually there is.. at least it helped in my case. i had a table with following struct: id = int word = varchar(15)
with about 500k rows, the initial query was mymodel.query.all() changing it to sequential load and limiting each query to 10k rows reduced the overall Mem usage from > 4GB to 250MB. On May 20, 9:51 pm, Adrian <[email protected]> wrote: > Ah, interesting! Geez, if that's the case there is no obvious > solution... > > On May 20, 7:54 am, "[email protected]" > > > > > > > > <[email protected]> wrote: > > Hi, > > i had something similar, the bottom line was that my query used too > > much RAM, thus forcing the swap go to overdrive. And the bottle neck > > was the read/write speed of the hard drive, preventing other treads to > > access swap. > > you can test that theory by monitoring your wa value in top or user > > iotop for finding out what is the highest consumer of hd bandwidth. > > > On May 18, 6:54 pm, Adrian <[email protected]> wrote: > > > > I've been working with Turbogears for a few months now, and one topic > > > that has always been a guess for me is how to properly handle > > > SQLAlchemy session objects. The webapp I'm working on now takes user > > > input and constructs SQLAlchemy queries that can take some time to > > > return. The way I currently have it structured is what I thought was > > > the "correct way." That is, I have the DBSession object defined in my > > > model, and in each controller I do: from webapp.model import > > > DBSession, session = DBSession(), session.query(..)... etc. I noticed > > > that if I go to a page and execute a long query, it hangs the whole > > > app. That is, if I (in a new tab) try to go to a page that is plain > > > HTML (I dropped it in public/test.html, so it doesn't even have an > > > associated controller), the HTML page doesn't load until after the > > > database query is complete. I guess I'm doing something wrong -- but > > > does anyone have some suggestions / advice? > > > > Now this next issue is a more general question. Even if I close the > > > page that is executing the long query, the webapp still hangs until it > > > is complete, at which point I get a broken pipe error from paster. > > > This is probably not possible, but is there any way to kill the > > > query / stop the controller from finishing if the user closes the > > > page? > > > > Thanks, > > > Adrian -- You received this message because you are subscribed to the Google Groups "TurboGears" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/turbogears?hl=en.
