For quick hits I usually turn debug logging on, or echo='debug' with create_engine(), that shows the rows coming in in the log.
There's no "per row" event interface, you can of course intercept execute events at the engine and cursor level with ConnectionProxy. A blunt approach may be to intercept cursor executes, ensure the statement is a SELECT, then run the statement distinctly (i.e. a second time) in the handler, count the rows, or at least do a fetchmany() of N number of rows and alert if over a certain threshold. Or wrap the query inside of "SELECT COUNT(*) FROM (<query>)" and get a count that way. Another depends on your DBAPI - if its observed that rows are pre-fetched within the execute, then you could just apply timing to a ConnectionProxy and look for slow queries (or large jumps in the size of gc.get_objects() maybe). Still another tack, subclass Query, override __iter__, pull out the result from super().__iter__(), count it, then return iter(result). Query by default buffers everything anyway. This would depend though on the fact that your query is returning distinct primary keys - if you have a basic cartesian product occurring (which is likely), Query's uniquifying of results might conceal that. On Mar 30, 2011, at 11:31 AM, Rick Morrison wrote: > Hi list: > > I've recently been plagued by a runaway query somewhere in one of my apps > that mistakenly loads 10s of 1000's of rows, swamping the working set of the > Python process and eventually invoking the OOM killer. > > Unfortunately, the database backend I'm using (MSSQL 2005) doesn't provide a > lot in the way of throttling or detection tools for this, and so I'd like to > inject some detection code into SQLA to track this thing down. Is there an > existing listener interface (or an appropriate injection location for some > code) in the (0.6.6) Engine or ResultProxy where it's possible to watch the > number of rows retrieved? > > Thanks, > Rick > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
