Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread Mike Bayer
s On Mon, May 14, 2018 at 9:38 PM, wrote: > The tid is coroutine_id, we get it by id(greenlet.getcurrent()) in each > coroutine(green thread). > > We run nova-conductor with monkey_patch, the same as [1], > eventlet.monkey_patch(os=False) > > Our nova-conductor is running with multi workers.

Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread jiajunsu . zju
The exc_info of tid 130446032 "Exception during reset or similar" is sqlalchemy.pool.QueuePool Traceback (most recent call last): sqlalchemy.pool.QueuePool File "sqlalchemy/pool.py", line 679, in _finalize_fairy sqlalchemy.pool.QueuePool fairy._reset(pool) sqlalchemy.pool.QueuePool File

Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread jiajunsu . zju
The tid is coroutine_id, we get it by id(greenlet.getcurrent()) in each coroutine(green thread). We run nova-conductor with monkey_patch, the same as [1], eventlet.monkey_patch(os=False) Our nova-conductor is running with multi workers. It's confusing why the _finalize_fairy is called twice bu

Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread jiajunsu . zju
I put the files we added logs here, include engine/base.py and pool.py https://github.com/jiajunsu/test_logs 在 2018年5月14日星期一 UTC+8下午11:10:37,Mike Bayer写道: > > OK I need the exact pool.py you are using as well that includes all > these logging statements and any other adjustments that were made.

Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread Mike Bayer
I notice in this log file there are multiple "tid", which I assume to be thread identifiers ? E.g. OS-level threads?Are you using eventlet or not?Are you monkeypatching SQLAlchemy and then using multiple OS-level threads at the same time? Because that likely doesn't work. Particularly, t

Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread Mike Bayer
OK I need the exact pool.py you are using as well that includes all these logging statements and any other adjustments that were made. On Mon, May 14, 2018 at 8:02 AM, wrote: > I've uploaded the logs here: > https://raw.githubusercontent.com/jiajunsu/test_logs/master/nc_5679.log > > The Connec

Re: [sqlalchemy] Re: ClauseAdapter.traverse alternative?

2018-05-14 Thread Mike Bayer
if you can provide an example of the specific operation you're doing I can evaluate if this might be impacting SQLAlchemy internally or otherwise what this use case is. On Wed, May 9, 2018 at 1:15 PM, Zac Goldstein wrote: > Thanks for the pointers and for checking. I misidentified the problem in

Re: [sqlalchemy] Re: long transaction after database switched over

2018-05-14 Thread jiajunsu . zju
I've uploaded the logs here: https://raw.githubusercontent.com/jiajunsu/test_logs/master/nc_5679.log The ConnectionRecord is 0x70cdb10, tid(coroutine id) is 130446032 At 2018-05-14T11:11:39.970, we got the log " jjs rec already in pool checkin", but I just deleted `return` in code, so the prob

Re: [sqlalchemy] trying to retrieve data from multiple tables with .join()

2018-05-14 Thread Simon King
On Mon, May 14, 2018 at 9:37 AM, wrote: > Hi, I'm still relatively a newby to SQLAlchemy and not sure if I'm using the > correct query to achieve what I'm trying to and hoping someone can point me > in the correct direction or help me understand what I need to use to run the > query I'm attemptin

[sqlalchemy] trying to retrieve data from multiple tables with .join()

2018-05-14 Thread chbrnola
Hi, I'm still relatively a newby to SQLAlchemy and not sure if I'm using the correct query to achieve what I'm trying to and hoping someone can point me in the correct direction or help me understand what I need to use to run the query I'm attempting. I'm trying to retrieve data from multiple t