Re: [sqlalchemy] Connection pool slow performance

2019-01-18 Thread Zsolt Ero
Thank you for the explanation, with ROLLBACK / pool_reset_on_return I understand it now. With this snippet: import time from sqlalchemy import create_engine e = create_engine("postgresql://postgres:sandbox@104.154.217.229/postgres") def go(): now = time.time() with e.connect() as

Re: [sqlalchemy] Connection pool slow performance

2019-01-18 Thread Zsolt Ero
But why is it doing any kind of network activity on a .connect(), if the previous connection was closed in the pool? OK, I created a sandbox server, it'll probably be quicker for you as the server is in US, but this also did 18 sec for me. import time from sqlalchemy import create_engine e =

Re: [sqlalchemy] Connection pool slow performance

2019-01-18 Thread Mike Bayer
On Fri, Jan 18, 2019 at 4:54 PM Mike Bayer wrote: > > On Fri, Jan 18, 2019 at 4:49 PM Zsolt Ero wrote: > > > > I know I'm far away from a remote server, as the server is a Google > > Cloud SQL instance, and I'm in a residential cable connection. But > > ping times are only 145 ms, nothing

Re: [sqlalchemy] Connection pool slow performance

2019-01-18 Thread Mike Bayer
On Fri, Jan 18, 2019 at 4:49 PM Zsolt Ero wrote: > > I know I'm far away from a remote server, as the server is a Google > Cloud SQL instance, and I'm in a residential cable connection. But > ping times are only 145 ms, nothing extreme I'd say. If you'd like I > can quickly setup a sandbox

Re: [sqlalchemy] Connection pool slow performance

2019-01-18 Thread Zsolt Ero
I know I'm far away from a remote server, as the server is a Google Cloud SQL instance, and I'm in a residential cable connection. But ping times are only 145 ms, nothing extreme I'd say. If you'd like I can quickly setup a sandbox instance on GCP for trying this out. Still, I don't even

Re: [sqlalchemy] Connection pool slow performance

2019-01-18 Thread Mike Bayer
On Fri, Jan 18, 2019 at 3:58 PM Zsolt Ero wrote: > > I thought I finally got to understand what does it mean to use a connection > pool / QueuePool, but this performance problem puzzles me. > > If I run: > for i in range(100): > with pg_engine.connect() as conn: >

[sqlalchemy] Connection pool slow performance

2019-01-18 Thread Zsolt Ero
I thought I finally got to understand what does it mean to use a connection pool / QueuePool, but this performance problem puzzles me. If I run: for i in range(100): with pg_engine.connect() as conn: conn.execute('select 1').fetchall() it takes 47 seconds. If I run with

Re: [sqlalchemy] Connection pool takes a long time to handle a closed connection

2018-10-25 Thread Sebastian Meinhardt
On Thursday, October 25, 2018 at 3:48:24 PM UTC+2, Mike Bayer wrote: > > > OK, so I believe your script should be using KILL SESSION, I've just > tried this and the effects are immediate: > > Yes, my own tests that I did in the meantime confirm this. As the original system is still down for

Re: [sqlalchemy] Connection pool takes a long time to handle a closed connection

2018-10-25 Thread Mike Bayer
On Thu, Oct 25, 2018 at 9:26 AM Sebastian Meinhardt wrote: > > Thank you for the advice. I will surely try out all the suggestions to > isolate the exact cause of the problem. Unfortunately, today of all days is > the annual maintenance and the system is down. I will try to reproduce the >

Re: [sqlalchemy] Connection pool takes a long time to handle a closed connection

2018-10-25 Thread Sebastian Meinhardt
Thank you for the advice. I will surely try out all the suggestions to isolate the exact cause of the problem. Unfortunately, today of all days is the annual maintenance and the system is down. I will try to reproduce the problem on a similar system. According to our database administrator,

Re: [sqlalchemy] Connection pool takes a long time to handle a closed connection

2018-10-25 Thread Mike Bayer
rops idle connections after an hour. My > > application uses the SQLAlchemy connection pool to deal with this. The > > problem is: When a connection is closed, it takes the pool a real long time > > (15-20 minutes) to recover. This happens with both strategies, > &g

Re: [sqlalchemy] Connection pool takes a long time to handle a closed connection

2018-10-25 Thread Simon King
On Thu, Oct 25, 2018 at 12:50 PM Sebastian Meinhardt wrote: > > Our database environment drops idle connections after an hour. My application > uses the SQLAlchemy connection pool to deal with this. The problem is: When a > connection is closed, it takes the pool a real long time (1

Re: [sqlalchemy] Connection pool takes a long time to handle a closed connection

2018-10-25 Thread Mike Bayer
On Thu, Oct 25, 2018 at 7:50 AM Sebastian Meinhardt wrote: > > Our database environment drops idle connections after an hour. My application > uses the SQLAlchemy connection pool to deal with this. The problem is: When a > connection is closed, it takes the pool a real long time (1

Re: [sqlalchemy] connection pool reconnect? (connection lost during postgres restart)

2014-10-11 Thread Jonathan Vanasco
great! i think everything is using a transaction, however one of my projects that uses the db models/engine config also has an engine that is non-transactional. i guess i'll have to test both, just to be sure -- You received this message because you are subscribed to the Google Groups

[sqlalchemy] connection pool reconnect? (connection lost during postgres restart)

2014-10-10 Thread Jonathan Vanasco
After reloading postgres's config, my web app had an error from a terminated PostgreSQL connection. After hitting reload a few times, the situation corrected itself ( there were multiple uwsgi workers with stale connections ). I started looking at the docs for dealing with disconnects (

Re: [sqlalchemy] connection pool reconnect? (connection lost during postgres restart)

2014-10-10 Thread Michael Bayer
On Oct 10, 2014, at 6:15 PM, Jonathan Vanasco jvana...@gmail.com wrote: After reloading postgres's config, my web app had an error from a terminated PostgreSQL connection. After hitting reload a few times, the situation corrected itself ( there were multiple uwsgi workers with stale

[sqlalchemy] Connection Pool behaviour change

2008-12-10 Thread Robert Ian Smit
Hi, I am porting our pylons app from 0.4.5 to 0.5. It appears that implicit queries are using a new connection instead of re-using the existing one that is also used by the ORM Session. I have read the Migration Docs and looked at the changelog and didn't find anything related to this matter.

[sqlalchemy] Sqlalchemy connection pool question

2008-03-25 Thread Ahmad Hassan
Hello, I am using sqlalchemy 4.0 in my application to connect to mysql database. I am running quries through session object which is instantiated as a (bind=engine, Transactional = True). I am not using threadlocal strategy. My question is that If I open 10 connection simultaneouslty then

[sqlalchemy] Re: Sqlalchemy connection pool question

2008-03-25 Thread Michael Bayer
On Mar 25, 2008, at 1:36 PM, Ahmad Hassan wrote: Hello, I am using sqlalchemy 4.0 in my application to connect to mysql database. I am running quries through session object which is instantiated as a (bind=engine, Transactional = True). I am not using threadlocal strategy. My

[sqlalchemy] connection pool

2007-03-14 Thread vkuznet
Hi, I just came across Documentation and it's not clear to me how to use connection pooling. When invoked db=create_engine() the pool parameter is set to None by default, right? In Connection pooling section of docs, it's said For most cases, explicit access to the pool module is not required