Re: [Zope-dev] Re: created z3c.saconfig
Am Freitag, 20. Juni 2008 13:59 schrieb Martijn Faassen: Hey, Hermann Himmelbauer wrote: [snip] 1) Why do you need to specify what interface the factory provides, such as here: component.provideUtility(engine_factory, provides=IEngineFactory) component.provideUtility(utility, provides=IScopedSession) Why can't the utilities provide the interface out of the box? They do, but then registration will only do the right thing if your utility only implements a single interface. I also think this makes the examples slightly easier to read. Anyway, it's not so important as normally registrations would take place from ZCML or using Grok. Ah, ok, I see. Perhaps it would be best to sketch the most simple case, with the bind parameter first, then explain what the shortcomings of this case are, and then introduce the engine utility. Yes, perhaps. I'm not sure whether that's a good idea in a tutorial; one that shows examples we don't want to encourage first, or the right example right away. Perhaps this could be done in an extra .txt file where we go into more detail about various options. +1 5) For the siteScopeFunc part, it would be best if there would already be a generic one in the SiteScopedSession class, although I don't know if this would be possible. However, this would make things simpler for beginners. Later on I suggest to explain that it's possible to overwrite this method and what it's for. I haven't found it easy for z3c.saconfig, as I tried to avoid dependencies on things like zope.traversing (which again pull in the world), or the ZODB. My intent is for z3c.saconfig to be foundational, but that other frameworks will need to fill in some more of the holes. My aim is to use this with megrok.rdb, and this will certainly offer a Grok-specific way to distinguish between applications. Ah, yes, dependency. Ok, then it should not be included out of the box. The missing bits in this module seem to be: 1) Some way to update database parameters, e.g. change your engine: In many web applications, database setup is done by the user during installation (e.g. PHProjekt and many others). The user has some install wizard and inputs the database parameters here, moreover he can change them later on via a web frontend. I think there should be some solution/guideline that aids the programmer in this part. I agree that this is still a feature that's missing and should be carefully tested. I'd like to avoid putting knowledge about user interfaces or the exact specification of SQLAlchemy parameters in z3c.saconfig though. I'd like to offer an infrastructure to reconfigure the engine and then make sure the reconfigured engine gets used, but only the minimal one. Again it's the task of applications or frameworks that build on top of this to use this infrastructure. What I can think of is: - Provide some updateEngine helper function, that queries a site/global registry for an IEngineUtility and alters the database parameters there - Somthing like I suggested in my code, where there are specific configuration properties in the utility, that, if changed, recreate the engine. Yes, I saw the property approach. I didn't want to go for the property approach, as I thought if you set 3 properties at once, the engine is disposed 3 times. Yes, true, that's not that pretty. However, this has no real bad consequences, I think. A simple updateEngineParameters function would also be an option, btw. What I want to offer is an API on IScopedSession and IEngineFactory that allows you to replace the engine parameters. This should dispose the engine, and then allow for a new engine to be created. I already have a sketch of the code on IEngineFactory, taken from your code. That isn't enough though, as the scoped session machinery will cache sessions and engines indefinitely. We also need an API on IScopedSession therefore that can trigger a particular engine (or set of engines) to be throw out from the scoped session. Yes, that looks nice. However, it has to be take special care that when changing the engine, open sessions are not somehow corrupted. Hm, do you have any details of how this can happen and can be avoided? I'm not sure about that. I just thought: What happens if someone actually uses a session and the engine is concurrently deleted/recreated from the EngineUtility? 2) Basically, I can think of 4 main scenarious for a Zope3 + SA integration: a) 1 database per Zope3 instance b) 1 database per Site c) n databases per Zope3 instance d) n databases per Zope3 Site I suggest to outline that in the beginning of the README.txt along with some introductory words and explain that the setup for these cases differ. I've added some text to the introduction and adjusted the headings for case a) and b). Perfect, right. (a) and (b) seem to be most common and are covered in z3c.saconfig, but (c)
Re: [Zope-dev] Re: created z3c.saconfig
Hey, On Sat, Jun 21, 2008 at 9:14 PM, Brian Sutherland [EMAIL PROTECTED] wrote: [snip] Ah, thanks for that tip. Does zope.app.cache.ram deal with threading issues? Ah, yes, I see it uses a lock. Anyway, a patch would be welcome. :) Committed, please feel free to revert or modify as you feel:) Thanks for this! Looks good! Looking at zope.app.cache.ram inspired me to think about a way to encapsulate all this storing-in-ram in a special kind of property. This should hopefully make this kind of code easer to write in the future. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: created z3c.saconfig
On Fri, Jun 20, 2008 at 02:01:52PM +0200, Martijn Faassen wrote: Hi there, Brian Sutherland wrote: [snip] Also for this problem: # XXX what happens if EngineFactory were to be evicted from the ZODB # cache? def getCached(self): return getattr(self, '_v_engine', None) I think you could use the same mechanism found in zope.app.cache.ram. I.e. store the engines in a module level global dictionary. Then use some clever way (with a counter and time) to figure out a unique key for your local utility (and persistently store the key). Ah, thanks for that tip. Does zope.app.cache.ram deal with threading issues? Ah, yes, I see it uses a lock. Anyway, a patch would be welcome. :) Committed, please feel free to revert or modify as you feel:) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: created z3c.saconfig
Am Freitag, 20. Juni 2008 00:14 schrieb Martijn Faassen: Hey, Martijn Faassen wrote: I intend to add support for a local utility soon, inspired by some code sent to me by Hermann Himmelbauer. This is now in there. It only looks faintly like Hermann's code, but it was still very useful. Thanks for the flowers. :-) This now works against the trunk of zope.sqlalchemy as well, as Laurence merged my branch. I do think my code currently requires SQLAlchemy 0.5beta1 or higher. Does it really? Because in the README.txt doctests, there seems to be a session.save() directive that is AFAIK deprecated by session.add(). Overall I'm quite pleased by this code. It's straightforward, and leaves to SQLAlchemy as much as possible, and just passes configuration parameters for engines and sessions along if you define them for the utilities. Yes, that's also my overall impression. It doesn't offer any user interface code or schemas itself; that's up to the application or framework developer. It's also flexible. Configuration information could be retrieved from any place a developer would like; hardcoded in Python, or the ZODB. I think it wouldn't be hard to write custom utilities that look up configuration in configuration files such as zope.conf or ZCML as well. In this part, I'd like to have something in the package as well, at least, some guidance of how to do this. Best Regards, Hermann -- [EMAIL PROTECTED] GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: created z3c.saconfig
Hermann Himmelbauer wrote: Am Freitag, 20. Juni 2008 00:14 schrieb Martijn Faassen: [snip] This now works against the trunk of zope.sqlalchemy as well, as Laurence merged my branch. I do think my code currently requires SQLAlchemy 0.5beta1 or higher. Does it really? Because in the README.txt doctests, there seems to be a session.save() directive that is AFAIK deprecated by session.add(). Interesting, I didn't get any deprecation warnings from SQLAlchemy. I'm using autocommit=False, and in the 0.4 era that's transactional=True. [snip] It doesn't offer any user interface code or schemas itself; that's up to the application or framework developer. It's also flexible. Configuration information could be retrieved from any place a developer would like; hardcoded in Python, or the ZODB. I think it wouldn't be hard to write custom utilities that look up configuration in configuration files such as zope.conf or ZCML as well. In this part, I'd like to have something in the package as well, at least, some guidance of how to do this. As long as it doesn't add a lot of dependencies. The package's dependencies are pretty small right now, and I tried to keep it that way. Even though I needed to test setSite() and such, I chose not to depend on zope.app.component as that pulls in the universe. Anyway, it could always be done in another package that builds on z3c.saconfig. Thanks for the feedback! Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: created z3c.saconfig
Hey, Hermann Himmelbauer wrote: [snip] 1) Why do you need to specify what interface the factory provides, such as here: component.provideUtility(engine_factory, provides=IEngineFactory) component.provideUtility(utility, provides=IScopedSession) Why can't the utilities provide the interface out of the box? They do, but then registration will only do the right thing if your utility only implements a single interface. I also think this makes the examples slightly easier to read. Anyway, it's not so important as normally registrations would take place from ZCML or using Grok. 2) I'd suggest to depict the case where an engine is bound to the session via the bind= parameter, as not all of us are that advanced in SA, thus it may be helpful. Moreover, you later on write that setting up an engine factory is not actually necessary, so the reader may ask himself why the engine utility makes sense at all. Yes, I wasn't sure about this one. Perhaps I should show this example, though it'll expand the example quite a bit and I'm not sure how it's helpful. I think we should encourage people to use the utility style registration for the engine. Perhaps it would be best to sketch the most simple case, with the bind parameter first, then explain what the shortcomings of this case are, and then introduce the engine utility. Yes, perhaps. I'm not sure whether that's a good idea in a tutorial; one that shows examples we don't want to encourage first, or the right example right away. Perhaps this could be done in an extra .txt file where we go into more detail about various options. 3) I'd suggest to explain the part, where you do a from z3c.saconfig import Session; session = Session() a little, and line out that it's used in SQLAlchemy style, e.g.: After registering the session utility, one can import the Session class vom z3c.saconfig, which offers the same capabilities as a common SQLAlchemy session class. You're right, that could use some more information. I've expanded the text. 4) In the site examples, it reads: sm1 = site1.getSiteManager() sm1.registerUtility(engine_factory1, provided=IEngineFactory) Why is it now provided instead of provides? Is this a typo or something specific? It's an annoying inconsistency in the zope.component APIs. It's not a typo. Again this is an API that at least Grok hides away for you. 5) For the siteScopeFunc part, it would be best if there would already be a generic one in the SiteScopedSession class, although I don't know if this would be possible. However, this would make things simpler for beginners. Later on I suggest to explain that it's possible to overwrite this method and what it's for. I haven't found it easy for z3c.saconfig, as I tried to avoid dependencies on things like zope.traversing (which again pull in the world), or the ZODB. My intent is for z3c.saconfig to be foundational, but that other frameworks will need to fill in some more of the holes. My aim is to use this with megrok.rdb, and this will certainly offer a Grok-specific way to distinguish between applications. The missing bits in this module seem to be: 1) Some way to update database parameters, e.g. change your engine: In many web applications, database setup is done by the user during installation (e.g. PHProjekt and many others). The user has some install wizard and inputs the database parameters here, moreover he can change them later on via a web frontend. I think there should be some solution/guideline that aids the programmer in this part. I agree that this is still a feature that's missing and should be carefully tested. I'd like to avoid putting knowledge about user interfaces or the exact specification of SQLAlchemy parameters in z3c.saconfig though. I'd like to offer an infrastructure to reconfigure the engine and then make sure the reconfigured engine gets used, but only the minimal one. Again it's the task of applications or frameworks that build on top of this to use this infrastructure. What I can think of is: - Simply reregister the engine utility with new parameters Hm, reregistering a local utility on the fly is rather hard-core, I'd like to avoid that and allow modification of engine utilities instead. - Provide some updateEngine helper function, that queries a site/global registry for an IEngineUtility and alters the database parameters there - Somthing like I suggested in my code, where there are specific configuration properties in the utility, that, if changed, recreate the engine. Yes, I saw the property approach. I didn't want to go for the property approach, as I thought if you set 3 properties at once, the engine is disposed 3 times. What I want to offer is an API on IScopedSession and IEngineFactory that allows you to replace the engine parameters. This should dispose the engine, and then allow for a new engine to be created. I already have a sketch of the code on IEngineFactory, taken
[Zope-dev] Re: created z3c.saconfig
Hi there, Brian Sutherland wrote: [snip] Also for this problem: # XXX what happens if EngineFactory were to be evicted from the ZODB # cache? def getCached(self): return getattr(self, '_v_engine', None) I think you could use the same mechanism found in zope.app.cache.ram. I.e. store the engines in a module level global dictionary. Then use some clever way (with a counter and time) to figure out a unique key for your local utility (and persistently store the key). Ah, thanks for that tip. Does zope.app.cache.ram deal with threading issues? Ah, yes, I see it uses a lock. Anyway, a patch would be welcome. :) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: created z3c.saconfig
Hey, Martijn Faassen wrote: I intend to add support for a local utility soon, inspired by some code sent to me by Hermann Himmelbauer. This is now in there. It only looks faintly like Hermann's code, but it was still very useful. You can register an engine factory globally or locally. This engine factory can be called to create an engine, but it'll return the same engine again when called again. This way, engines are shared and engine pooling works. To make this work for your application/framework you need to subclass SiteScopedSession and implement siteScopeFunc. You can then register an instance of that subclass globally (or locally if you should so desire, but globally is usually enough). The README was updated to reflect the new situation: http://svn.zope.org/z3c.saconfig/trunk/src/z3c/saconfig/README.txt This now works against the trunk of zope.sqlalchemy as well, as Laurence merged my branch. I do think my code currently requires SQLAlchemy 0.5beta1 or higher. Overall I'm quite pleased by this code. It's straightforward, and leaves to SQLAlchemy as much as possible, and just passes configuration parameters for engines and sessions along if you define them for the utilities. It doesn't offer any user interface code or schemas itself; that's up to the application or framework developer. It's also flexible. Configuration information could be retrieved from any place a developer would like; hardcoded in Python, or the ZODB. I think it wouldn't be hard to write custom utilities that look up configuration in configuration files such as zope.conf or ZCML as well. One thing I'm not sure about is using _v_engine to store the engine in a persistent local utility. What if the local utility gets evicted from the ZODB cache? It'd need to recreate the engine. Anyway, I've left the policy on how to cache the engine overridable. I'm curious about better ideas. I considered using a global dictionary with the cached engines in there, as nothing could get just disappear from that, but I'm not entirely sure that'd be thread safe. There is a chance that an engine gets recreated if two threads were to be writing to it at the same time. Perhaps some thread locking code is required. Opinions? Feedback is again welcome! Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
