On Jun 6, 2014, at 11:52 AM, Bill Schindler <[email protected]> wrote:
> I'm chasing a problem where our app slowly uses more and more db connections. > QueuePool eventually logs a TimeoutError because it has run out of available > connections. (SA 0.9.4) > > The app has a background task that wakes up and checks the db for any pending > work. Most of time, the check-out/return-to-pool is what I'd expect (grab 3 > connections from the pool, use them, return them). But once in a while, the > QueuePool decides to create a new connection. This happens even though > there's connections that have been returned to the pool. Here's an example: > > sqlalchemy.pool.QueuePool Connection <connection object at 0x47cb340; dsn: > 'dbname=iia user=postgres host=lspdbsrvr03.localdomain port=5432', closed: 0> > checked out from pool > sqlalchemy.pool.QueuePool Connection <connection object at 0x4498fe0; dsn: > 'dbname=iia user=postgres host=lspdbsrvr03.localdomain port=5432', closed: 0> > checked out from pool > sqlalchemy.pool.QueuePool Connection <connection object at 0x4498fe0; dsn: > 'dbname=iia user=postgres host=lspdbsrvr03.localdomain port=5432', closed: 0> > being returned to pool > sqlalchemy.pool.QueuePool Connection <connection object at 0x4498fe0; dsn: > 'dbname=iia user=postgres host=lspdbsrvr03.localdomain port=5432', closed: 0> > rollback-on-return, via agent > sqlalchemy.pool.QueuePool Connection <connection object at 0x47cb340; dsn: > 'dbname=iia user=postgres host=lspdbsrvr03.localdomain port=5432', closed: 0> > being returned to pool > sqlalchemy.pool.QueuePool Connection <connection object at 0x47cb340; dsn: > 'dbname=iia user=postgres host=lspdbsrvr03.localdomain port=5432', closed: 0> > rollback-on-return, via agent > sqlalchemy.pool.QueuePool Created new connection <connection object at > 0x490e190; dsn: 'dbname=iia user=postgres host=lspdbsrvr03.localdomain > port=5432', closed: 0> > > What would cause the QueuePool to allocate a new connection when there's > obviously connections available in the pool? > > It's the same task loop running over-and-over (the task is not even getting > new work), so there shouldn't be anything changing there. QueuePool allocates a new connection when you call connect() and there's no more available connections in the pool. It will go until it has (pool size + max_overflow) connections known to exist and then it will start blocking, unless you have max_overflow = -1 in which case it will go without stopping. As far as it creating new connections when there *are* connections present, the two cases that can cause that are if you have pool_recycle set to a positive value, or if a connection were invalidated due to certain exceptions which indicate a connection reset, or if invalidation or connection detach were called explicitly. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
