Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-19 Thread Jonathan Vanasco
On Tuesday, October 18, 2016 at 9:29:37 AM UTC-4, Mike Bayer wrote: This approach can also be made extremely performant by making sure you > load up that profile datafile just once, and cache it in memory in a > dictionary. > What's also nice about the decorator approach in the aspect, is

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-18 Thread Mike Bayer
On 10/17/2016 11:33 PM, Alfred Perlstein wrote:ith the community. Adding a snippet like this to my app was very helpful, however I'm wondering if there's something I'm missing that may cause a catastrophe? I'm looking at maybe making it useful for detecting if a new piece of code happens to

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-17 Thread Alfred Perlstein
I was not offended personally, however I was annoyed that a conversation on a topic that I felt helpful to the community was being shut down. When I shared the response I got the feedback was that the received answer fell under was the umbrella of "mansplainining" and how a response such as

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-17 Thread Jonathan Vanasco
Alfred- I'm sorry you felt offended and misinterpreted my comment. I did not think you were unskilled in the slightest. Quite the opposite (i did look you up). Your concerns and solutions were in-line with to how every highly experienced lower-level engineer I've worked with has approached

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-15 Thread Alfred Perlstein
On 10/12/16 3:45 PM, Jonathan Vanasco wrote: On Wednesday, October 12, 2016 at 5:58:41 PM UTC-4, Alfred Perlstein wrote: Oh nice!! Will this be called in the context of the same thread executing the sql? This would allow me to debug without formatting all debugs I

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Mike Bayer
On 10/12/2016 05:49 PM, Alfred Perlstein wrote: Mike, Thank you, in fact thank you very, very much. Writing out the code was really above and beyond my expectations. I will use this snippet in our app. The following problems remain for us (and also likely other users of sqla) although

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Jonathan Vanasco
On Wednesday, October 12, 2016 at 5:58:41 PM UTC-4, Alfred Perlstein wrote: > > Oh nice!! Will this be called in the context of the same thread > executing the sql? This would allow me to debug without formatting all > debugs I believe... > It depends on how you're using threads and

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Alfred Perlstein
On 10/12/16 1:51 AM, Simon King wrote: You could use the before_cursor_execute event to log the SQL based on some debugging flag. http://docs.sqlalchemy.org/en/latest/core/events.html#sqlalchemy.events.ConnectionEvents.before_cursor_execute For example: from sqlalchemy import event

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Alfred Perlstein
On 10/12/16 10:44 AM, Mike Bayer wrote: On 10/12/2016 02:18 AM, Mike Bayer wrote: In this case I'd in the simple case just add "threadid" to the log format (https://docs.python.org/2/library/logging.html#logrecord-attributes) and use grep against the stream. More ambitiously I'd write a

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Mike Bayer
On 10/12/2016 02:18 AM, Mike Bayer wrote: In this case I'd in the simple case just add "threadid" to the log format (https://docs.python.org/2/library/logging.html#logrecord-attributes) and use grep against the stream. More ambitiously I'd write a simple logging.Filter that only allows

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Mike Bayer
On 10/12/2016 09:51 AM, Mike Bayer wrote: On 10/12/2016 03:13 AM, Alfred Perlstein wrote: Mike, I'm trying to find a nice way to say that this doesn't totally make life difficult for us. Nestled deep in your message I can see you had a concern about performance, and perhaps you thought

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Mike Bayer
On 10/12/2016 03:13 AM, Alfred Perlstein wrote: Mike, I'm trying to find a nice way to say that this doesn't totally make life difficult for us. Hi Alfred - I'd point out that you've spent the time and energy here to produce about eight paragraphs describing how disappointed you'd be if

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Simon King
You could use the before_cursor_execute event to log the SQL based on some debugging flag. http://docs.sqlalchemy.org/en/latest/core/events.html#sqlalchemy.events.ConnectionEvents.before_cursor_execute For example: from sqlalchemy import event DEBUGGING = False @event.listens_for(SomeEngine,

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Alfred Perlstein
Mike, I'm trying to find a nice way to say that this doesn't totally make life difficult for us. Even if we were to use the filter as described, we couldn't use it for an existing connection, we would HAVE to restart the connection? Does that really sound reasonable to you to force users

Re: [sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-12 Thread Mike Bayer
On 10/11/2016 09:59 PM, Alfred Perlstein wrote: Hello folks, we are using SqlA in a pretty large flask app. One issue we have is that sometimes due to orm, serializers (marshmallow) or other complex code paths we find ourselves doing many more queries than we expected and this causes us a lot

[sqlalchemy] Selective logging using the ORM for a single thread. Can't do it, therefore can't debug our app.

2016-10-11 Thread Alfred Perlstein
Hello folks, we are using SqlA in a pretty large flask app. One issue we have is that sometimes due to orm, serializers (marshmallow) or other complex code paths we find ourselves doing many more queries than we expected and this causes us a lot of headache. We need to be able to track down