Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Fri, May 30, 2008 at 01:55:44PM -0400, Benji York wrote: > On Fri, May 30, 2008 at 1:40 PM, Brian Sutherland > <[EMAIL PROTECTED]> wrote: > > I've just decided to jettison IAbsoluteURL and make a new interface > > ICanonicalURL. The adapters for ICanonicalURL are available anywhere > > without specially wrapping the object in a location proxy. > > IAbsoluteURL doesn't neccesarily require a location proxy, it's just that the > default implementations require objects to implement ILocation. > > No need to define your own interface, just override the IAbsoluteURL > implementation(s) you don't like to behave differently (or better yet, don't > let them get registered in the first place). One thing I don't like about the default implementation is that if you override IAbsoluteURL for a container, it has no effect for contained objects. I suppose there's a good reason for that -- efficiency -- but it often makes custom IAbsoluteURL adapters pointless. LocationProxy helps then. Marius Gedminas -- Photons have energy, and trying to cram too many into too small of a space can cause a black hole to form, which is, needless to say, not a desirable trait for an optical computer. -- http://scottaaronson.com/blog/?p=261#comment-13693 signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Fri, May 30, 2008 at 3:03 PM, Martijn Faassen <[EMAIL PROTECTED]> wrote: > Benji York wrote: >> I have an app that allows the same object to be reachable via several >> URLs. When we wanted one of the URLs to be preferred (none are really >> "cononical" in my case), then we did just as you suggest above and added >> a way for the IAbsoluteURL adapter(s) to know which URL was preferred. >> It worked out quite well. > > Cool. Where does the adapter get the information to make a decision from? I > mean, how does it know to use URL A and not URL B? Nothing fancy: we adapt the object to (names changed to clarify the example) IPreferredLocations to get back a list ordered by how preferred the locations are (from most-preferred to least), we then simply iterate over the list (it's always short) until we find the first place the object is visible, and use it. The location ordering is specified by the user (more or less, there are details I'm leaving out). -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On May 30, 2008, at 9:44 AM, Martijn Faassen wrote: Hey, Michael Bayer wrote: [snip discussion on BoundMetadata I don't comprehend yet but probably should :)] As far as ScopedSession, it's really just a thread local variable. Like any global threadlocal, you can stick whatever you want on it at the start of a request, and tear it down at the end. Is it really needed to set things up at the start and the request and tear things down on the end, or can a session be retained between requests? I.e. is the intent to recreate a Session class each time a request is issued? in the 0.4 series, the Session is "sticky" in that whatever is in it retains the state that was last loaded - 0.4 does not go back to the database to re-fetch the state of an object unless its explicitly told to do so. When an HTTP request ends, assuming all objects in the session are marked as "clean", they're weakly referenced and will fall out of scope assuming those objects were only referenced by the request. So in that sense, you can just leave the Session alone and it'll work just fine for the next request...but any pending changes in the Session that weren't flushed for some reason would get carried over, as well as anything that might be strongly referenced elsewhere. So we always recommended in 0.4 to at least issue a session.clear() at the end of a request to just empty it out (but you can still reuse that session). Other options included scopedsession.remove() which tears the whole session down, the advantage there being that the new request could configure the next session differently (as in, binding it to something else). in 0.5, Session has been changed to be less reluctant to go and "re- fetch" data (which is contrary to the particular background I came from, but since then I've learned to see a broader scope of use cases). In 0.5, after a commit() or rollback(), the Session still may be holding on to objects, but their state is expired such that it will all be re-loaded upon access, and all "pending" and "deleted" states are reverted. So 0.5's Session, when configured in the default way, makes it very hard to get at "stale" state, so in that sense you can just re-use a session wihtout worrying much about what may have been left over. Then again it's also reasonable that you might want to have individual ScopedSessions for each application instance within a multi-app process, that way the burden of setting up/tearing down is reduced or removed. This indicates to me is possible to retain a Session object between two requests? I.e. it's a worthwhile goal to reduce the amount of Session configuration going on, right? Its not a strong argument either way to reuse a session or just make a new one on each request. In 0.4, making a brand new session on each request does have a "cleaner" feel to it since theres no chance of any state left hanging around. Its not an expensive operation. A single ScopedSession for a multi-app process is like a one- dimensional approach where both current thread and current app are identified by the current "get()" of the registry; a collection of ScopedSessions is more like a two-dimensional approach where the first level of registry (i.e. which ScopedSession do I choose) distinguishes between app instance, and the second level (i.e. what Session is bound to this thread ID) distinguishes between threads. All of that said I think it can work either way but I think the latter approach might have the explicitness you're looking for. I'm trying to understand why you think so. I am looking for a way to let developers use SQLAlchemy in a straightforward way. They should be able to import 'Session', instantiate session in their app, and everything works as expected. The framework takes care of making you get the appropriately configured Session. they can in fact do this without any issue.The question is, when someone writes the call "s = Session()" three times within one request, do you want each "s" to all reference the *same* set of objects ? that was the issue scoped_session() was meant to solve. Configuration is easy since you just configure a "Session" callable for that application. But you guys have this issue of "multiple applications in the same process" at play, which each talk to a different database. So thats where a decision has to be made how to deal with that complexity, either configure engine on one scoped_session per request, or configure multiple scoped_sessions, one per engine. The latter approach seems "easier" to me since you don't actually have to do anything on a per-request basis assuming 0.5 usage. Neither of these make any difference to the end user who sees the exact same usage pattern. (I'd still absolutely like to avoid passing in context explicitly each time you need a session; it puts unnecessary burden on t
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Benji York wrote: On Fri, May 30, 2008 at 2:35 PM, Martijn Faassen <[EMAIL PROTECTED]> wrote: I guess one difference here is that it seems more common in the RDB scenario to have multiple ways to reach the same object, and even if you just publish one, you can get to the actual object in multiple ways. In this case it'd be nice to convince IAbsoluteURL to somehow know which URL option is the best, or alternatively, give the object somehow its 'canonical' location (parent, etc) even if you don't get it that way. I have an app that allows the same object to be reachable via several URLs. When we wanted one of the URLs to be preferred (none are really "cononical" in my case), then we did just as you suggest above and added a way for the IAbsoluteURL adapter(s) to know which URL was preferred. It worked out quite well. Cool. Where does the adapter get the information to make a decision from? I mean, how does it know to use URL A and not URL B? Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Fri, May 30, 2008 at 2:35 PM, Martijn Faassen <[EMAIL PROTECTED]> wrote: > I guess one difference here is that it seems more common in the RDB scenario > to have multiple ways to reach the same object, and even if you just publish > one, you can get to the actual object in multiple ways. > > In this case it'd be nice to convince IAbsoluteURL to somehow know which URL > option is the best, or alternatively, give the object somehow its > 'canonical' location (parent, etc) even if you don't get it that way. I have an app that allows the same object to be reachable via several URLs. When we wanted one of the URLs to be preferred (none are really "cononical" in my case), then we did just as you suggest above and added a way for the IAbsoluteURL adapter(s) to know which URL was preferred. It worked out quite well. -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Brian Sutherland wrote: [snip] This was a problem with sqlos containers as well. It gets nasty quickly. Say you have 2 containers, of people and addresses. Also people can have an address. So you can get to an address in 2 different ways: /adresses/address1 /people/person1/address1 What should the url for address1 be? I'd vote for /adresses/address1. But due to the LocationProxy, you'll get different results for different IAbsoluteURL calls depending on how you traversed to the object. I've seen this complaint coming from quite a few people a number of times, so this needs some thinking about. One interesting observation is that if you do this in the ZODB you'd get the exact same problem, and then people use things like zc.shortcut to get around it. So, in the ZODB, we typically don't give the same object two URLs. I guess one difference here is that it seems more common in the RDB scenario to have multiple ways to reach the same object, and even if you just publish one, you can get to the actual object in multiple ways. In this case it'd be nice to convince IAbsoluteURL to somehow know which URL option is the best, or alternatively, give the object somehow its 'canonical' location (parent, etc) even if you don't get it that way. This is the "it should work like ZODB-backed objects" pattern I've been trying to follow as a design guideline. Since for contained objects I can always get the URL, it should work that way for RDB-based contained objects too. I'm not sure it's worthwhile following the "it should work like ZODB-backed objects" pattern. The nice property of ZODB-backed objects is that they're just plain Python objects, and it's also familiar to me, so that's why I look for special reasons to diverge from it. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Fri, May 30, 2008 at 1:40 PM, Brian Sutherland <[EMAIL PROTECTED]> wrote: > I've just decided to jettison IAbsoluteURL and make a new interface > ICanonicalURL. The adapters for ICanonicalURL are available anywhere > without specially wrapping the object in a location proxy. IAbsoluteURL doesn't neccesarily require a location proxy, it's just that the default implementations require objects to implement ILocation. No need to define your own interface, just override the IAbsoluteURL implementation(s) you don't like to behave differently (or better yet, don't let them get registered in the first place). -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Fri, May 30, 2008 at 03:49:46PM +0200, Martijn Faassen wrote: > Brian Sutherland wrote: > [snip] >> I'll note that there is a difference between where things are published >> (IAbsoluteURL) and where you should go to find their canonical >> representation. >> >> One problem I've found with using IAbsoluteURL and location proxies is >> getting the object directly from the database. In this situation you >> don't have an LocationProxy and IAbsoluteURL doesn't work. >> >> It's pretty nasty when IAbsoluteURL sometimes works and sometimes >> doesn't for what essentially is the same object. > > Agreed, we should ideally always get our objects from the db already > properly wrapped in a location proxy (or providing ILocation directly). I > think we can do this with containers based on relations; those containers > are under our control after all. I've just decided to jettison IAbsoluteURL and make a new interface ICanonicalURL. The adapters for ICanonicalURL are available anywhere without specially wrapping the object in a location proxy. > Is a direct one on one relation possible in SQLAlchemy? I.e. an object foo > that has an attribute 'bar' that is another ORM-ed object. If so we also > need to take care of that case somehow. This was a problem with sqlos containers as well. It gets nasty quickly. Say you have 2 containers, of people and addresses. Also people can have an address. So you can get to an address in 2 different ways: /adresses/address1 /people/person1/address1 What should the url for address1 be? I'd vote for /adresses/address1. But due to the LocationProxy, you'll get different results for different IAbsoluteURL calls depending on how you traversed to the object. > This is the "it should work like ZODB-backed objects" pattern I've been > trying to follow as a design guideline. Since for contained objects I can > always get the URL, it should work that way for RDB-based contained objects > too. I'm not sure it's worthwhile following the "it should work like ZODB-backed objects" pattern. > > Regards, > > Martijn > > ___ > Zope-Dev maillist - Zope-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Brian Sutherland wrote: [snip] I'll note that there is a difference between where things are published (IAbsoluteURL) and where you should go to find their canonical representation. One problem I've found with using IAbsoluteURL and location proxies is getting the object directly from the database. In this situation you don't have an LocationProxy and IAbsoluteURL doesn't work. It's pretty nasty when IAbsoluteURL sometimes works and sometimes doesn't for what essentially is the same object. Agreed, we should ideally always get our objects from the db already properly wrapped in a location proxy (or providing ILocation directly). I think we can do this with containers based on relations; those containers are under our control after all. Is a direct one on one relation possible in SQLAlchemy? I.e. an object foo that has an attribute 'bar' that is another ORM-ed object. If so we also need to take care of that case somehow. This is the "it should work like ZODB-backed objects" pattern I've been trying to follow as a design guideline. Since for contained objects I can always get the URL, it should work that way for RDB-based contained objects too. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Hey, Michael Bayer wrote: [snip discussion on BoundMetadata I don't comprehend yet but probably should :)] As far as ScopedSession, it's really just a thread local variable. Like any global threadlocal, you can stick whatever you want on it at the start of a request, and tear it down at the end. Is it really needed to set things up at the start and the request and tear things down on the end, or can a session be retained between requests? I.e. is the intent to recreate a Session class each time a request is issued? Then again it's also reasonable that you might want to have individual ScopedSessions for each application instance within a multi-app process, that way the burden of setting up/tearing down is reduced or removed. This indicates to me is possible to retain a Session object between two requests? I.e. it's a worthwhile goal to reduce the amount of Session configuration going on, right? A single ScopedSession for a multi-app process is like a one-dimensional approach where both current thread and current app are identified by the current "get()" of the registry; a collection of ScopedSessions is more like a two-dimensional approach where the first level of registry (i.e. which ScopedSession do I choose) distinguishes between app instance, and the second level (i.e. what Session is bound to this thread ID) distinguishes between threads. All of that said I think it can work either way but I think the latter approach might have the explicitness you're looking for. I'm trying to understand why you think so. I am looking for a way to let developers use SQLAlchemy in a straightforward way. They should be able to import 'Session', instantiate session in their app, and everything works as expected. The framework takes care of making you get the appropriately configured Session. That underneath sessions are scoped per thread and per app, and configured per app, is something the developer who handles the session object ideally should not have to worry about. Since I hope we can actually make this work, why do you think an explicit approach with a custom registry which then contains scoped sessions would be better? The less of our own new registries we invent the better, I'd say. That said, I guess you could make the same semantics work if you make Session be some factory that uses a nested registry. (I'd still absolutely like to avoid passing in context explicitly each time you need a session; it puts unnecessary burden on the developer and promises to make multi-application interactions easier while it actually doesn't do so) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Thu, May 29, 2008 at 07:10:16PM +0200, Martijn Faassen wrote: > [snip] > > I'm not sure whether it would be a good idea to wrap this in a session > > property, or just register it as an adapter. The only other object that > > would need access to the session (either as a property or through > > adaption) would be the application instance root object. Something like: > > > > @adapter(MyApp) > > @provider(ISession) > > def root_session(context): > > return context._sessioncontext() > > This looks to me like it could be a simple function that looks up a local > utility instead: > > def session(): >return component.getUtility(ISessionContext)() > > We get the right context from the site that way. I don't see the point in > trying to re-implement local utilities with adapters while zope.component > already does something quite similar for you. That said, I still have hope > we can used ScopedSession and forgo a utility lookup here, see below... +lots Perhaps you may want the utility name as a keyword. I'll also note that doing it this way does not preclude using the ISession adapter in future. But, using the ISession adapter now means never being able to get rid of it again. > If you don't use the ZODB at all, you could still set up local sites (I'm > not sure how hard this is, but it *should* be possible; I've done it as a work around to override a global utility forced upon me. It's not too difficult. > > We do still need to setup parent pointers though for grok.url and > > IAbsoluteURL to work. It looks fairly easy to add location information > > to the containers themselves: > > (note that grok.url uses IAbsoluteURL, so we just care about IAbsoluteURL) I'll note that there is a difference between where things are published (IAbsoluteURL) and where you should go to find their canonical representation. One problem I've found with using IAbsoluteURL and location proxies is getting the object directly from the database. In this situation you don't have an LocationProxy and IAbsoluteURL doesn't work. It's pretty nasty when IAbsoluteURL sometimes works and sometimes doesn't for what essentially is the same object. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On May 28, 2008, at 7:09 PM, Laurence Rowe wrote: Engine configuration is a subset of session configuration. You cannot have a single ScopedSession for a package if you want to have multiple instances of that application. We must work with unbound metadata if we have this goal. That implies that we must use bound sessions, a session associated with a particular engine (actually it could be more complex than this, you can associate particular classes/tables with particular engines within a session). I would agree that "bound" metadata is a little cumbersome in an environment that wants to switch engines within a single process. "Bound" metadata has always given us a lot of headaches because it's just *so* convenient that we just can't get rid of it, but at the same time people are just so tripped up by it, thinking it's necessary to make anything happen (it's not). Binding sessions allows better control of connection/transaction scope (since the Session relates to transactions, MetaData does not), so it's probabably the way to go in an environment that has a lot of things going on. As far as ScopedSession, it's really just a thread local variable. Like any global threadlocal, you can stick whatever you want on it at the start of a request, and tear it down at the end. Then again it's also reasonable that you might want to have individual ScopedSessions for each application instance within a multi-app process, that way the burden of setting up/tearing down is reduced or removed. A single ScopedSession for a multi-app process is like a one-dimensional approach where both current thread and current app are identified by the current "get()" of the registry; a collection of ScopedSessions is more like a two-dimensional approach where the first level of registry (i.e. which ScopedSession do I choose) distinguishes between app instance, and the second level (i.e. what Session is bound to this thread ID) distinguishes between threads. All of that said I think it can work either way but I think the latter approach might have the explicitness you're looking for. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Hey Laurence, [cc-ed Mike in case he hadn't seen this thread and has comments] Thanks for the continued back and forth; I'm learning about SQLAlchemy and hopefully my pushing helps you improve your ideas too. Laurence Rowe wrote: Martijn Faassen wrote: [using scoped sessions] That would be fine if you had the same configuration across all sessions (e.g. they all connected to the same engine / database) or each session was configured at the start of every request. Presumably we will want to connect different application instances to different databases. Mike Bayer suggested configuring the session with the proper engine at the start of each request in his example that I quote at the start of the thread. How expensive actually is it to configure a session with an engine? I'm still getting back to combining scoped sessions and utilities, though. We need to do two things: * convince scoped session to return the session appropriate for our application. We can do this by introducing a custom scopefunc that takes application into account (by looking up a utility in it that knows what scope the application has) * convince the session factory to look up application specific configuration information the first time a new scope is entered. This means that if we have a central register of ScopedSessions, then we must key it by some unique application id (path I guess). def application_session(app): try: return registry[app.getPath()]() except KeyError: return registry.setdefault(app.getPath(), scoped_session(sessionmaker(**app.getConfiguration(() I don't understand, why is a further registry necessary? What is wrong with the modified scopefunc I suggested (using path or whatever you prefer to key it)? Then you don't need a registry of ScopedSessions, ScopedSession *is* that registry... def our_scope_func(): return (component.getUtility(ISessionSiteScope).applicationScope(), thread.getindent()) where applicationScope could be the path or the unique id or whatever would be best for the application, as long as it's unique per app. Then we also introduce a custom session factory that does a utility lookup for its configuration (I'm handwaving hopefully non-essential details here): def our_session_factory(): config = component.getUtility(ISessionSiteScope).configuration() return sessionmaker(**config)() Session = scoped_session(our_session_factory, our_scope_func) ScopedSession should then take care of the rest. ISessionSiteScope is looked up each time you instantiate Session. 'configuration()' is only called the *first* time you instantiate a session in a particular scope, after that the configuration is cached. My point about using adapters, or indeed properties to access the session, is that the only object needing to access the session which cannot look it up directly with Session.object_session(object) is the application root object. To me it seems simpler to do this than to register utilities. Also it would be nice to have a consistent way to lookup the session. I think we can agree that the ideal way to look up the session would be to allow the following in applications, right? from zope.rdbintegration import Session session = Session() and then session would always be an appropriately configured session for the current scope. Now please explain to me why what I sketched out above can't work. :) (note that if you want two cooperating applications each using their own session, you're required to use setSite() before you call into the second application. This is required anyway to make other local utility lookups work properly for that other application, so this is not an extra burden on developers) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Martijn Faassen wrote:
Laurence Rowe wrote:
Martijn Faassen wrote:
[snip]
Before we talk more about session configuration, please explain why
we're not talking about engine configuration. :)
Engine configuration is a subset of session configuration. You cannot
have a single ScopedSession for a package if you want to have
multiple instances of that application.
I don't understand then what Mike Bayer wrote before:
Mike Bayer wrote:
If you are running different instances each connected to a different
engine within one process you wouldn't need any awareness of engines
at the object level (therefore no entity_name) and also no engine
proxying - you should have separate Session instances for each
"process" managed by scoped_session(), which was designed to handle
this.Multiple apps on one codebase within one process was an
original requirement of Pylons as well, though nobody has ever used
it.
That seemed to suggest to me that scoped sessions were an appropriate
solution.
Anyway, back to you:
[snip]
> I'm not sure whether it would be a good idea to wrap this in a session
> property, or just register it as an adapter. The only other object that
> would need access to the session (either as a property or through
> adaption) would be the application instance root object. Something like:
>
> @adapter(MyApp)
> @provider(ISession)
> def root_session(context):
> return context._sessioncontext()
This looks to me like it could be a simple function that looks up a
local utility instead:
def session():
return component.getUtility(ISessionContext)()
We get the right context from the site that way. I don't see the point
in trying to re-implement local utilities with adapters while
zope.component already does something quite similar for you. That said,
I still have hope we can used ScopedSession and forgo a utility lookup
here, see below...
> And the simplest persistent session context might be:
>
> class PersistentSessionContext(persistent):
> implements(ISessionContext)
>
> def __init__(self, url, twophase=False, engine_kw={}, session_kw={}):
> self.url = url
> self.twophase = twophase
> self.engine_kw = engine_kw
> self.session_kw = session_kw
>
> def __call__(self):
> session = getattr(self._v_session, None)
> if session is None:
> engine = getattr(self._v_engine, None)
> if engine is None:
> engine = self._v_engine = create_engine(
> self.url, **self.engine_kw)
> session = self._v_session = create_session(
> bind=engine, twophase=self.twophase, **self.session_kw)
> return session
Doesn't ScopedSession already take care of this using ScopedRegistry?
>
Perhaps a more clever scopefunc is necessary to introduce a per-instance
scope? Right now it's only per-thread scope. scopefunc could also do a
local utility lookup that gets it a way to uniquely identify the current
application (not sure what would be best, object id? zodb id? a unique
string generated for each installed application?).
Something like this:
def scopefunc():
return (component.getUtility(ISessionSiteScope).applicationScope(),
thread.getindent())
Session = scoped_session(sessionmaker(autoflush=True), scopefunc)
> A more complex scheme might maintain a dict of ScopedSessions keyed by
> application path, outside of the object cache. You could also ensure
> that only a single engine is created for a given set of arguments, but
> this seems overkill
> Everything would then get a session consistently with a call to
> ISession(self) or ISession(self.context) in the case of views. No parent
> pointers involved.
I still don't understand why this is nicer than a local utility lookup.
I understand the two cooperating applications use case, but that can be
easily provided for with using setSite(), just like you *already* need
to do to make everything work right with the other local utilities that
this application might be using. Above in my scopefunc example I assume
that setSite has been set up appropriately.
That would be fine if you had the same configuration across all sessions
(e.g. they all connected to the same engine / database) or each session
was configured at the start of every request. Presumably we will want to
connect different application instances to different databases.
This means that if we have a central register of ScopedSessions, then we
must key it by some unique application id (path I guess).
def application_session(app):
try:
return registry[app.getPath()]()
except KeyError:
return registry.setdefault(app.getPath(),
scoped_session(sessionmaker(**app.getConfiguration(()
My point about using adapters, or indeed properties to access the
session, is that the only object needing to access the session which
cannot look it up directly with Session.object_session(object) is the
application root object. To me it seems simpler to do this than to
r
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Laurence Rowe wrote:
Martijn Faassen wrote:
[snip]
Before we talk more about session configuration, please explain why
we're not talking about engine configuration. :)
Engine configuration is a subset of session configuration. You cannot
have a single ScopedSession for a package if you want to have
multiple instances of that application.
I don't understand then what Mike Bayer wrote before:
Mike Bayer wrote:
If you are running different instances each connected to a different
engine within one process you wouldn't need any awareness of engines
at the object level (therefore no entity_name) and also no engine
proxying - you should have separate Session instances for each
"process" managed by scoped_session(), which was designed to handle
this.Multiple apps on one codebase within one process was an
original requirement of Pylons as well, though nobody has ever used
it.
That seemed to suggest to me that scoped sessions were an appropriate
solution.
Anyway, back to you:
[snip]
> I'm not sure whether it would be a good idea to wrap this in a session
> property, or just register it as an adapter. The only other object that
> would need access to the session (either as a property or through
> adaption) would be the application instance root object. Something like:
>
> @adapter(MyApp)
> @provider(ISession)
> def root_session(context):
> return context._sessioncontext()
This looks to me like it could be a simple function that looks up a
local utility instead:
def session():
return component.getUtility(ISessionContext)()
We get the right context from the site that way. I don't see the point
in trying to re-implement local utilities with adapters while
zope.component already does something quite similar for you. That said,
I still have hope we can used ScopedSession and forgo a utility lookup
here, see below...
> And the simplest persistent session context might be:
>
> class PersistentSessionContext(persistent):
> implements(ISessionContext)
>
> def __init__(self, url, twophase=False, engine_kw={}, session_kw={}):
> self.url = url
> self.twophase = twophase
> self.engine_kw = engine_kw
> self.session_kw = session_kw
>
> def __call__(self):
> session = getattr(self._v_session, None)
> if session is None:
> engine = getattr(self._v_engine, None)
> if engine is None:
> engine = self._v_engine = create_engine(
> self.url, **self.engine_kw)
> session = self._v_session = create_session(
> bind=engine, twophase=self.twophase, **self.session_kw)
> return session
Doesn't ScopedSession already take care of this using ScopedRegistry?
Perhaps a more clever scopefunc is necessary to introduce a per-instance
scope? Right now it's only per-thread scope. scopefunc could also do a
local utility lookup that gets it a way to uniquely identify the current
application (not sure what would be best, object id? zodb id? a unique
string generated for each installed application?).
Something like this:
def scopefunc():
return (component.getUtility(ISessionSiteScope).applicationScope(),
thread.getindent())
Session = scoped_session(sessionmaker(autoflush=True), scopefunc)
> A more complex scheme might maintain a dict of ScopedSessions keyed by
> application path, outside of the object cache. You could also ensure
> that only a single engine is created for a given set of arguments, but
> this seems overkill
> Everything would then get a session consistently with a call to
> ISession(self) or ISession(self.context) in the case of views. No parent
> pointers involved.
I still don't understand why this is nicer than a local utility lookup.
I understand the two cooperating applications use case, but that can be
easily provided for with using setSite(), just like you *already* need
to do to make everything work right with the other local utilities that
this application might be using. Above in my scopefunc example I assume
that setSite has been set up appropriately.
If you don't use the ZODB at all, you could still set up local sites
(I'm not sure how hard this is, but it *should* be possible;
zope.component has no knowledge of persistence), or if it's just one app
per zope instance, set up a global ISessionSiteScope utility.
> We do still need to setup parent pointers though for grok.url and
> IAbsoluteURL to work. It looks fairly easy to add location information
> to the containers themselves:
(note that grok.url uses IAbsoluteURL, so we just care about IAbsoluteURL)
So far this isn't a particular problem in Grok as traversal uses
zope.location.located() to locate everything. That said, if you want to
directly access an object by accessing ORM-mapped attributes and then
get a URL for that object, this won't work. Since it *does* work when
you use the ZODB, it'd be nice if it also worked properly with sqlalchemy.
Hopefully we can indeed make containers behave the right way by making
our own Map
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Martijn Faassen wrote: Laurence Rowe wrote: We need to differentiate between the interface for session configuration and session usage from an application. Session configuration? I'm talking about engine configuration. A session doesn't need to be configured, except with a session, I think, which the ScopedSession machinery allows you to do. If you can make the engine be the right one in the appropriate context it should only be a matter of configuring the session in that context. For session usage I think it is fairly simple. We should define an ISessionContext interface such that: class ISessionContext(Interface): def __call__(): "return a session in this context" A future version of collective.lead could implement an ISessionContext. Client code however should have a cleaner interface, a plain ISession. This is accessed through a lookup on the context, translated into a simple adapter: def session_adapter(context): session_context = queryUtility(ISessionContext, context=context, default=None) if session_context is not None: return session_context() This will allow application code to do something like: session = ISession(context) ob = session.query(MyClass) I don't understand what the point of doing this is. What is 'context' here? Why am I adapting context? My context is normally thread-local and implicit. This has been the way to approach context in Zope 3 for a long time now (though explicit context can still be passed through getUtility it's rarely done). And again, I think *engine* should be in context, not sessions. ScopedSession, a standard SQLAlchemy mechanism, should be used for session context-specific session access. Why the introduction of ISessionContext and ISession interfaces, and an adapter lookup that looks up a utility and then *still* I haven't seen the code that actually configures the engine? My aim was to try to stick to SQLAlchemy patterns for solving this problem where we had no reason to diverge from them, and our use case, I take it, is what ScopedSession was designed for. [snip] session.remove() is not important, sessions are closed by the zope.sqlalchemy datamanager and closed sessions are recyclable. That's good to seee confirmed. I thought it was that way reading the code, but I wanted to make sure. Presumably the session object would be referred to by a volatile attribute on the local utility and the session would be GC'd along with the local utility object itself. Are you talking about a persistent local utility? Which one? Would this mean that the session needs to be re-created each time the ZODB swaps out the object with the volatile attribute? I thought the session was intended to be recreated each request, does it make any sense to cache them between requests? Table creation is another matter that I don't think too important. I think it's important to get it right for Grok. We're using the declarative extension and still want to support hand-created tables as well. There are various scenarios surrounding table creation, either not doing it at all ever, or spelling them out by hand in Python, or by inlining them into the classes as with the declarative extension. Implicit creation of tables seems wrong, instead tables should only be created explicitly, by the use clicking a button in the zope web interface (or automatically on adding an application). Automatic on adding an application is a good point to do it, as that way we don't bother the person who creates the application too much (first create the database.. then install the application, then go to this screen and create the tables). There's another feature to automatic table creation though: when I'm developing and I don't care about the data yet, all I need to do now when I change the schema is throw away the database and create it again. I found this very convenient while developing with collective.lead. We need to have something that is at least as convenient for this use case (common during initial development). An exception to this is sqlalchemy in memory databases, which must be created on first access. Don't know what these are? Those with a url like 'sqlite://:memory:' Session configuration would be somewhat similar to collective.lead currently (registering one as a local utility). Before we talk more about session configuration, please explain why we're not talking about engine configuration. :) Engine configuration is a subset of session configuration. You cannot have a single ScopedSession for a package if you want to have multiple instances of that application. We must work with unbound metadata if we have this goal. That implies that we must use bound sessions, a session associated with a particular engine (actually it could be more complex than this, you can associate particular classes/tables with particular engines within a session). The simplest way to achiev
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Brian Sutherland wrote: On Sat, May 24, 2008 at 09:30:18PM +0100, Laurence Rowe wrote: Brian Sutherland wrote: On Fri, May 23, 2008 at 11:39:39PM +0100, Laurence Rowe wrote: We need to differentiate between the interface for session configuration and session usage from an application. For session usage I think it is fairly simple. We should define an ISessionContext interface such that: class ISessionContext(Interface): def __call__(): "return a session in this context" +lots (I was thinking about proposing an interface called ISessionMaker doing much the same thing) I'm not sure what "in this context" means? The context of this application instance. Say you have two instances of an application, one pointed at database A, another at database B. It is possible to involve both applications in a single request / transaction. You must be sure that you are working with the correct session. Are the instances you are talking about persistent objects in the ZODB? Yes, I'm assuming that the application instances live in the ZODB and are local site managers for this example. A future version of collective.lead could implement an ISessionContext. Client code however should have a cleaner interface, a plain ISession. This is accessed through a lookup on the context, translated into a simple adapter: def session_adapter(context): session_context = queryUtility(ISessionContext, context=context, Why call queryUtility with the context keyword? default=None) if session_context is not None: return session_context() This will allow application code to do something like: session = ISession(context) ob = session.query(MyClass) This really confuses me. What is the context? Does it have any meaning? Or is it just a shorter way to write: session = getUtility(ISessionContext)() Does the value of context have an effect on what you get from the ISession adaptation? Yes, as it translates directly to the getUtility lookup. It ensures that lookups in /appA go to a local utility defined in /appA and lookups in /appB go to a local utility defined in /appB. I've been burned by using context as a keyword to getUtility before. When your context doesn't have a proper __parent__ pointer, the default IComponentLookup adapter falls back to the global site manager and ignores the local one. That causes no end of confusion and hard to debug bugs as people will call ISession on objects with and without __parent__ pointers and then wonder why it fails. Perhaps a more robust way is to rely on the site stored while traversing in zope.app.component.hooks? For example: * appA implements ISite and has a local ISessionContext utility * The path appA is traversed over (i.e. the url path is /appA/foo) * queryUtility(ISessionContext) then returns the local utility from appA without the need to specify a "context" (thanks to the magic in zope.app.component.hooks) I think the only situation where you really want to specify a "context" for queryUtility is when you want to access a local utility registered in /appB from a URL path like /appA/foo. That seems pretty rare and almost broken. You could implement a function like: def query_session(name=u''): return queryUtility(ISessionContext, name=name)() That does the right thing in almost all cases (i.e. uses the closest local site), and is much more robust. I fear that if we rely on the site manager set during traversal, then applications will rely on that when looking up their session in application code, and it will be impossible to involve objects from different application instances in the same transaction. getUtility already looks up in the global component registry if a site manager cannot be found in the context's parents (or where __parent__ is missing). I guess this concept could be extended with something like: def session_adapter(context): smgr = getSiteManager(context) if smgr is getGlobalSiteManager(): smgr = None return getUtility(ISessionContext, context=smgr)() This would defer the bugs to the point you start working with two instances of an application in a single request. I wouldn't consider accessing objects from /appA and /appB in the same request broken, for someone coming from a ZODB background it would seems quite reasonable. Of course it would be possible to register a ScopedSession globally as such a utility, but more usually a local utility should be registered. Depends what you're doing. If you are running without a ZODB, you have mostly just global utilities. It would be a pity if zope.sqlalchemy started to depend on the ZODB. Wihout ZODB and zope.app.component it seems unlikely that you would be able to register a local utility, the idea of ISessionContext is so that you might be able to register a ScopedSession as a global utility too. I very much like the idea of ISessionContext! I'm just not sure about how you suggest using it
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Sat, May 24, 2008 at 09:30:18PM +0100, Laurence Rowe wrote: > Brian Sutherland wrote: >> On Fri, May 23, 2008 at 11:39:39PM +0100, Laurence Rowe wrote: >>> We need to differentiate between the interface for session configuration >>> and session usage from an application. >>> >>> For session usage I think it is fairly simple. We should define an >>> ISessionContext interface such that: >>> >>> class ISessionContext(Interface): >>> def __call__(): >>> "return a session in this context" >> >> +lots >> >> (I was thinking about proposing an interface called ISessionMaker doing >> much the same thing) >> >> I'm not sure what "in this context" means? > > The context of this application instance. Say you have two instances of an > application, one pointed at database A, another at database B. It is > possible to involve both applications in a single request / transaction. > You must be sure that you are working with the correct session. Are the instances you are talking about persistent objects in the ZODB? >>> A future version of collective.lead could implement an ISessionContext. >>> Client code however should have a cleaner interface, a plain ISession. >>> This is accessed through a lookup on the context, translated into a >>> simple adapter: >>> >>> def session_adapter(context): >>> session_context = queryUtility(ISessionContext, context=context, >> >> Why call queryUtility with the context keyword? >> >>> default=None) >>> if session_context is not None: >>> return session_context() >>> >>> This will allow application code to do something like: >>> >>> session = ISession(context) >>> ob = session.query(MyClass) >> >> This really confuses me. What is the context? Does it have any meaning? >> Or is it just a shorter way to write: >> >> session = getUtility(ISessionContext)() >> >> Does the value of context have an effect on what you get from the >> ISession adaptation? > > Yes, as it translates directly to the getUtility lookup. It ensures that > lookups in /appA go to a local utility defined in /appA and lookups in > /appB go to a local utility defined in /appB. I've been burned by using context as a keyword to getUtility before. When your context doesn't have a proper __parent__ pointer, the default IComponentLookup adapter falls back to the global site manager and ignores the local one. That causes no end of confusion and hard to debug bugs as people will call ISession on objects with and without __parent__ pointers and then wonder why it fails. Perhaps a more robust way is to rely on the site stored while traversing in zope.app.component.hooks? For example: * appA implements ISite and has a local ISessionContext utility * The path appA is traversed over (i.e. the url path is /appA/foo) * queryUtility(ISessionContext) then returns the local utility from appA without the need to specify a "context" (thanks to the magic in zope.app.component.hooks) I think the only situation where you really want to specify a "context" for queryUtility is when you want to access a local utility registered in /appB from a URL path like /appA/foo. That seems pretty rare and almost broken. You could implement a function like: def query_session(name=u''): return queryUtility(ISessionContext, name=name)() That does the right thing in almost all cases (i.e. uses the closest local site), and is much more robust. >> >>> Of course it would be possible to register a ScopedSession globally as >>> such a utility, but more usually a local utility should be registered. >> >> Depends what you're doing. If you are running without a ZODB, you have >> mostly just global utilities. >> >> It would be a pity if zope.sqlalchemy started to depend on the ZODB. > > Wihout ZODB and zope.app.component it seems unlikely that you would be able > to register a local utility, the idea of ISessionContext is so that you > might be able to register a ScopedSession as a global utility too. I very much like the idea of ISessionContext! I'm just not sure about how you suggest using it in client code. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zope.sqlalchemy, integration ideas
Brian Sutherland wrote: On Fri, May 23, 2008 at 11:39:39PM +0100, Laurence Rowe wrote: We need to differentiate between the interface for session configuration and session usage from an application. For session usage I think it is fairly simple. We should define an ISessionContext interface such that: class ISessionContext(Interface): def __call__(): "return a session in this context" +lots (I was thinking about proposing an interface called ISessionMaker doing much the same thing) I'm not sure what "in this context" means? The context of this application instance. Say you have two instances of an application, one pointed at database A, another at database B. It is possible to involve both applications in a single request / transaction. You must be sure that you are working with the correct session. A future version of collective.lead could implement an ISessionContext. Client code however should have a cleaner interface, a plain ISession. This is accessed through a lookup on the context, translated into a simple adapter: def session_adapter(context): session_context = queryUtility(ISessionContext, context=context, Why call queryUtility with the context keyword? default=None) if session_context is not None: return session_context() This will allow application code to do something like: session = ISession(context) ob = session.query(MyClass) This really confuses me. What is the context? Does it have any meaning? Or is it just a shorter way to write: session = getUtility(ISessionContext)() Does the value of context have an effect on what you get from the ISession adaptation? Yes, as it translates directly to the getUtility lookup. It ensures that lookups in /appA go to a local utility defined in /appA and lookups in /appB go to a local utility defined in /appB. Of course it would be possible to register a ScopedSession globally as such a utility, but more usually a local utility should be registered. Depends what you're doing. If you are running without a ZODB, you have mostly just global utilities. It would be a pity if zope.sqlalchemy started to depend on the ZODB. Wihout ZODB and zope.app.component it seems unlikely that you would be able to register a local utility, the idea of ISessionContext is so that you might be able to register a ScopedSession as a global utility too. (I haven't though about the consequences of this in pre-traversal, before the site and local utilities are set up) session.remove() is not important, sessions are closed by the zope.sqlalchemy datamanager and closed sessions are recyclable. Presumably the session object would be referred to by a volatile attribute on the local utility and the session would be GC'd along with the local utility object itself. Table creation is another matter that I don't think too important. Implicit creation of tables seems wrong, instead tables should only be created explicitly, by the use clicking a button in the zope web interface (or automatically on adding an application). An exception to this is sqlalchemy in memory databases, which must be created on first access. Session configuration would be somewhat similar to collective.lead currently (registering one as a local utility). Laurence Martijn Faassen wrote: Hi there, Today I had a discussion with Jasper Spaans about how to go about improving megrok.rdb, Grok's relational database integration which aims to integrate Grok with SQLAlchemy. We started developing megrok.rdb at the Grokkerdam sprint a few weeks ago. We reread the discussion surrounding zope.sqlalchemy for ideas on how to go about integration and configuration. I think these discussions reach wider than just Grok's concerns. Note that I'm not proposing we fold any of these ideas into zope.sqlalchemy itself, which should remain as free of policy as possible; it could become (yet another) extension. Let me run our current thinking by the list. What would be nice in Zope applications (and we think would be good for Grok) would be per-instance database configuration. That is, we want to be able to install multiple instances of the same application and then configure each of them with a different database URN and it should all work, each talking to their own database. Michael Bayer's suggestion involves the use of scoped sessions. He proposed the following code: Session = scoped_session() # start of request engine = get_appropriate_engine() Session(bind=engine) try: # do request finally: Session.remove() Let's go through the steps. First it makes a scoped session object, it then configures it with the right engine at the start of the request (it can do this on a per-class level), and then at the end of the request it removes the Session again, which results in the actual session being closed. Our get_appropriate_engine() would probably look the engine up as a local utility, as Laurence suggested.
Re: [Zope-dev] Re: zope.sqlalchemy, integration ideas
On Fri, May 23, 2008 at 11:39:39PM +0100, Laurence Rowe wrote: > We need to differentiate between the interface for session configuration > and session usage from an application. > > For session usage I think it is fairly simple. We should define an > ISessionContext interface such that: > > class ISessionContext(Interface): > def __call__(): > "return a session in this context" +lots (I was thinking about proposing an interface called ISessionMaker doing much the same thing) I'm not sure what "in this context" means? > A future version of collective.lead could implement an ISessionContext. > Client code however should have a cleaner interface, a plain ISession. This > is accessed through a lookup on the context, translated into a simple > adapter: > > def session_adapter(context): > session_context = queryUtility(ISessionContext, context=context, Why call queryUtility with the context keyword? > default=None) > if session_context is not None: > return session_context() > > This will allow application code to do something like: > > session = ISession(context) > ob = session.query(MyClass) This really confuses me. What is the context? Does it have any meaning? Or is it just a shorter way to write: session = getUtility(ISessionContext)() Does the value of context have an effect on what you get from the ISession adaptation? > Of course it would be possible to register a ScopedSession globally as such > a utility, but more usually a local utility should be registered. Depends what you're doing. If you are running without a ZODB, you have mostly just global utilities. It would be a pity if zope.sqlalchemy started to depend on the ZODB. > (I haven't though about the consequences of this in pre-traversal, before > the site and local utilities are set up) > > session.remove() is not important, sessions are closed by the > zope.sqlalchemy datamanager and closed sessions are recyclable. Presumably > the session object would be referred to by a volatile attribute on the > local utility and the session would be GC'd along with the local utility > object itself. > > Table creation is another matter that I don't think too important. Implicit > creation of tables seems wrong, instead tables should only be created > explicitly, by the use clicking a button in the zope web interface (or > automatically on adding an application). An exception to this is sqlalchemy > in memory databases, which must be created on first access. > > Session configuration would be somewhat similar to collective.lead > currently (registering one as a local utility). > > Laurence > > Martijn Faassen wrote: >> Hi there, >> >> Today I had a discussion with Jasper Spaans about how to go about >> improving megrok.rdb, Grok's relational database integration which aims to >> integrate Grok with SQLAlchemy. We started developing megrok.rdb at the >> Grokkerdam sprint a few weeks ago. We reread the discussion surrounding >> zope.sqlalchemy for ideas on how to go about integration and >> configuration. I think these discussions reach wider than just Grok's >> concerns. Note that I'm not proposing we fold any of these ideas into >> zope.sqlalchemy itself, which should remain as free of policy as possible; >> it could become (yet another) extension. >> >> Let me run our current thinking by the list. >> >> What would be nice in Zope applications (and we think would be good for >> Grok) would be per-instance database configuration. That is, we want to be >> able to install multiple instances of the same application and then >> configure each of them with a different database URN and it should all >> work, each talking to their own database. >> >> Michael Bayer's suggestion involves the use of scoped sessions. He >> proposed the following code: >> >> Session = scoped_session() >> >> # start of request >> engine = get_appropriate_engine() >> Session(bind=engine) >> try: >> # do request >> finally: >> Session.remove() >> >> Let's go through the steps. First it makes a scoped session object, it >> then configures it with the right engine at the start of the request (it >> can do this on a per-class level), and then at the end of the request it >> removes the Session again, which results in the actual session being >> closed. >> >> Our get_appropriate_engine() would probably look the engine up as a local >> utility, as Laurence suggested. There is a bit of question about engine >> configuration, though. >> >> If we want to support the use case of looking up the engine URL in a >> persistent datastore (for instance one URL per location), we have a >> question of ordering. We cannot do it too early; at the start of the >> transaction there isn't a ZODB yet to talk to so we can't look up a local >> utility. We can try doing it just in time: >> >> _Session = scoped_session() >> >> def Session(*args, **kw): >> engine = get_appropriate_engine() >> _Session.bind(bind=engi
[Zope-dev] Re: zope.sqlalchemy, integration ideas
We need to differentiate between the interface for session configuration and session usage from an application. For session usage I think it is fairly simple. We should define an ISessionContext interface such that: class ISessionContext(Interface): def __call__(): "return a session in this context" A future version of collective.lead could implement an ISessionContext. Client code however should have a cleaner interface, a plain ISession. This is accessed through a lookup on the context, translated into a simple adapter: def session_adapter(context): session_context = queryUtility(ISessionContext, context=context, default=None) if session_context is not None: return session_context() This will allow application code to do something like: session = ISession(context) ob = session.query(MyClass) Of course it would be possible to register a ScopedSession globally as such a utility, but more usually a local utility should be registered. (I haven't though about the consequences of this in pre-traversal, before the site and local utilities are set up) session.remove() is not important, sessions are closed by the zope.sqlalchemy datamanager and closed sessions are recyclable. Presumably the session object would be referred to by a volatile attribute on the local utility and the session would be GC'd along with the local utility object itself. Table creation is another matter that I don't think too important. Implicit creation of tables seems wrong, instead tables should only be created explicitly, by the use clicking a button in the zope web interface (or automatically on adding an application). An exception to this is sqlalchemy in memory databases, which must be created on first access. Session configuration would be somewhat similar to collective.lead currently (registering one as a local utility). Laurence Martijn Faassen wrote: Hi there, Today I had a discussion with Jasper Spaans about how to go about improving megrok.rdb, Grok's relational database integration which aims to integrate Grok with SQLAlchemy. We started developing megrok.rdb at the Grokkerdam sprint a few weeks ago. We reread the discussion surrounding zope.sqlalchemy for ideas on how to go about integration and configuration. I think these discussions reach wider than just Grok's concerns. Note that I'm not proposing we fold any of these ideas into zope.sqlalchemy itself, which should remain as free of policy as possible; it could become (yet another) extension. Let me run our current thinking by the list. What would be nice in Zope applications (and we think would be good for Grok) would be per-instance database configuration. That is, we want to be able to install multiple instances of the same application and then configure each of them with a different database URN and it should all work, each talking to their own database. Michael Bayer's suggestion involves the use of scoped sessions. He proposed the following code: Session = scoped_session() # start of request engine = get_appropriate_engine() Session(bind=engine) try: # do request finally: Session.remove() Let's go through the steps. First it makes a scoped session object, it then configures it with the right engine at the start of the request (it can do this on a per-class level), and then at the end of the request it removes the Session again, which results in the actual session being closed. Our get_appropriate_engine() would probably look the engine up as a local utility, as Laurence suggested. There is a bit of question about engine configuration, though. If we want to support the use case of looking up the engine URL in a persistent datastore (for instance one URL per location), we have a question of ordering. We cannot do it too early; at the start of the transaction there isn't a ZODB yet to talk to so we can't look up a local utility. We can try doing it just in time: _Session = scoped_session() def Session(*args, **kw): engine = get_appropriate_engine() _Session.bind(bind=engine) return _Session(*args, **kw) Here get_appropriate_engine() could do a component.getUtility() and look up the engine for us, possibly in an application-local way. There's still the question of how this engine got configured in the first place. How does it know the database URL? How does the engine get created after the database URL is known (this might be quite late in the game; it could be stored in the ZODB). It then starts to look more and more attractive to do something similar like collective.lead's IDatabase utility, which can be stored persistently in the ZODB and has a getEngine() method which actually gets the engine (creating it if necessary). If we use sqlalchemy.ext.declarative, we also need to make the declarative extension of SQLALchemy load up the tables at the right point in time. We would also like a way to hook into matters and register som
