On Mar 24, 2011, at 10:26 AM, Andrew wrote:
> We have an application using SQLAlchemy to query an Oracle database.
> We started to see errors with the QueuePool running out of
> connections, so we upped the number of connections (and overflow).
> However, now we're seeing behavior where, when running in mod_wsgi,
> the number of Apache processes goes wild, and all the httpd processes
> freeze. There are *no* error messages except for the following:
>
> [Wed Mar 23 19:02:01 2011] [error] Exception AttributeError:
> AttributeError("'NoneType' object has no attribute 'pop'",) in <bound
> method InstanceState._cleanup of <sqlalchemy.orm.state.InstanceState
> object at 0x2aaab4448f50>> ignored
>
> I saw Michael's post (http://groups.google.com/group/sqlalchemy/
> browse_thread/thread/744b333985f14d50/f9a586bc8a68e4ec?
> lnk=gst&q=AttributeERror+_cleanup&fwc=2) about this, indicating its
> not an issue _assuming that the Session is cleaned up_ properly. A
> good portion of the code uses:
>
> conn = Session.connection()
> cursor = conn.connection.cursor()
>
> # Several of the following
> cursor.execute("...")
> cursor.callproc("...")
>
> There's no explicit closing of the Session or the cursor itself; would
> this cause the GC issues we're seeing, and the eventual hanging of
> Apache?
dealing with the cursor in that way is probably not harmful though its a good
idea to call close() on it when you're done with it. The "ignored" warning is
entirely irrelevant to the cursor issue, it has to do with a large number of
objects being abruptly garbage collected, as when a child fork is being shut
down for example. Its true that if your sessions were explicitly closed out
at the end you shouldn't see much of those. Then also, the httpd processes
freezing is likely not related to either of those things, but that's your big
problem which is that your app is deadlocking. A common cause of this has to
do with open transactions conflicting table or row locks against each other,
but it also occurs if you hit the limit on QueuePool, as it will block once the
total number of connections + overflow is exhausted.
So it seems like your main issue is that your app doesn't close out
connections. You want to ensure that your web requests are unconditionally
framed inside a try/finally block which establishes connection state at the
start, then tears it down at the end. For Session, the options are close(),
rollback(), or commit(). close() is safest since it detaches all objects,
eliminating the possibility that one of them might be accessed again and start
a new transaction. remove() is also the best if you're using a ScopedSession
(which is typical for web applications).
There's some guidelines at
http://www.sqlalchemy.org/docs/orm/session.html#lifespan-of-a-contextual-session
. Also you want to make sure you don't have requests that for some reason
are just hanging open (like if you're playing with comet or something like
that, the configuration would be dramatically different).
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
