On 12/16/2015 04:07 PM, Kai Groner wrote:
> Hi,
>
> I work on a project that uses dependency injection. We want to log
> queries in the context of a particular instantiation of the injector.
> This means that when we register an event listener, that listener has
> references to a unique logger configuration and we don't want that
> listener to leak into other contexts.
>
> I think the way to do this is to use a separate Engine for each
> injector, but I still want to have a shared connection pool. My first
> attempt looked similar to this:
>
> _pool = None
> @Injector.factory('db_engine')
> def engine_factory():
> nonlocal _pool
> engine = create_engine(dburi, pool=_pool)
> if _pool is None:
> _pool = engine.pool
> return engine
oh, OK. This is, a little bit brittle. Maybe not as much as in
earlier releases but an engine.dispose() call is still going to make you
sad.
>
> This worked. Some of the engine events are actually pool events, so
> those couldn't be isolated, but I was able to avoid using those. This
> also leaks memory (I think it's related to the new dialect object each
> time we create an engine). Perhaps this isn't how connection pools were
> meant to be used.
hmmm, memory leaks, not sure, but then again no, we assume the pool is
owned by a single Engine.
>
> Now I'm considering another way to isolate the engine and it's event
> listeners. It looks like this:
>
> _engine = None
> @Injector.factory('db_engine')
> def engine_factory():
> nonlocal _engine
> if _engine is None:
> _engine = create_engine(dburi)
> engine = copy.copy(_engine)
> engine.dispatch = copy.copy(engine.dispatch)
> return engine
that looks like something that would totally break. the event dispatch
system is extremely meta / dynamic / weak referenced / special and a
blunt tool like copy(), I can't imagine what that would do.
>
> This works ok, and it's leaking less memory than it was before.
OK well it's not like we're trying to seal with duct tape, any leak
means, "it's broke" :)
But,
> I've made some assumptions that I probably shouldn't be making.
>
> So, I'm looking for comments or advice on whether this is a terrible
> idea, or if there's another way I should be approaching this.
You can make event-independent "proxies" of engines using the
engine.execution_options() feature. You can define whatever keywords
you want there (or not send any), and you can make event listeners that
are local to that engine and alternatively can peek into the
_execution_options dictionary (which means you could really just have
one listener that dispatches based on something in the dictionary).
from sqlalchemy import create_engine, event
e = create_engine("sqlite://", echo=True)
def listen_for_thing(keyword, engine):
@event.listens_for(engine, "before_cursor_execute")
def before_cursor_execute(
conn, cursor, statement, parameters, context, executemany):
print("statement for engine %s / %s: '%s'" % (
keyword, conn._execution_options['our_engine_name'], statement))
e1 = e.execution_options(our_engine_name='engine1')
e2 = e.execution_options(our_engine_name='engine2')
listen_for_thing('engine1', e1)
listen_for_thing('engine2', e2)
e1.execute("select 'engine1'")
e2.execute("select 'engine2'")
>
>
> Thanks,
> Kai
>
> --
> You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected]
> <mailto:[email protected]>.
> To post to this group, send email to [email protected]
> <mailto:[email protected]>.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
