Just putting this on record...
I found where the problem was. I had defined a "baseQuery" in
app_globals:
class Globals(object):
def baseQuery(self, session):
return session.query(Stuff.stuff1, Stuff.stuff2, Stuff.stuff3).
\
join(Things, OtherStuff, Derp)
This was what was holding up the rest of the app -- once I moved this
query definition to each individual controller that uses it, long db
queries no longer hang the site.
On May 20, 6:51 pm, "[email protected]"
<[email protected]> wrote:
> actually there is.. at least it helped in my case.
> i had a table with following struct:
> id = int
> word = varchar(15)
>
> with about 500k rows, the initial query was mymodel.query.all()
> changing it to sequential load and limiting each query to 10k rows
> reduced the overall Mem usage from > 4GB to 250MB.
>
> On May 20, 9:51 pm, Adrian <[email protected]> wrote:
>
>
>
> > Ah, interesting! Geez, if that's the case there is no obvious
> > solution...
>
> > On May 20, 7:54 am, "[email protected]"
>
> > <[email protected]> wrote:
> > > Hi,
> > > i had something similar, the bottom line was that my query used too
> > > much RAM, thus forcing the swap go to overdrive. And the bottle neck
> > > was the read/write speed of the hard drive, preventing other treads to
> > > access swap.
> > > you can test that theory by monitoring your wa value in top or user
> > > iotop for finding out what is the highest consumer of hd bandwidth.
>
> > > On May 18, 6:54 pm, Adrian <[email protected]> wrote:
>
> > > > I've been working with Turbogears for a few months now, and one topic
> > > > that has always been a guess for me is how to properly handle
> > > > SQLAlchemy session objects. The webapp I'm working on now takes user
> > > > input and constructs SQLAlchemy queries that can take some time to
> > > > return. The way I currently have it structured is what I thought was
> > > > the "correct way." That is, I have the DBSession object defined in my
> > > > model, and in each controller I do: from webapp.model import
> > > > DBSession, session = DBSession(), session.query(..)... etc. I noticed
> > > > that if I go to a page and execute a long query, it hangs the whole
> > > > app. That is, if I (in a new tab) try to go to a page that is plain
> > > > HTML (I dropped it in public/test.html, so it doesn't even have an
> > > > associated controller), the HTML page doesn't load until after the
> > > > database query is complete. I guess I'm doing something wrong -- but
> > > > does anyone have some suggestions / advice?
>
> > > > Now this next issue is a more general question. Even if I close the
> > > > page that is executing the long query, the webapp still hangs until it
> > > > is complete, at which point I get a broken pipe error from paster.
> > > > This is probably not possible, but is there any way to kill the
> > > > query / stop the controller from finishing if the user closes the
> > > > page?
>
> > > > Thanks,
> > > > Adrian
--
You received this message because you are subscribed to the Google Groups
"TurboGears" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/turbogears?hl=en.
