Re: [Zope-CMF] Move to github?
Hi, On 3 March 2013 18:45, Andreas Jung li...@zopyx.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 yuppie wrote: You do realise it's: a) free (for us) b) decentralised What do you mean by it? What by free? What is decentralised? I mean, there's no tangible cost (financial or otherwise) of using GitHub; and git's architecture pretty much ensures that there's no lock-in (especially if mirroring is set up). Why do your points a) and b) make supporting GitHub Inc. a good decision? I don't see it as supporting GitHub. I see it as using a service that is free to us and rather good. It saves resources (e.g. the time spent managing svn.zope.org; the cost of bandwidth) that can be better spent elsewhere (e.g. working on Zope/CMF). It helps make it easier for others to contribute, because so many people already know how to use GitHub. GitHub Inc. is too successful. It already has too much power. That's not good for the open source community. Because? We all value your contributions to Zope and CMF __very much__ but is it really necessary being that fundamental? I'd echo that sentiment (especially the first part). What's the worst that could happen? GitHub goes belly-up and we starting using a different remote in our repos? GitHub tries to violate the license terms of our software somehow (that seems very unlikely)? Martin ___ Zope-CMF maillist - Zope-CMF@zope.org https://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] Move to github?
On 2 March 2013 16:18, yuppie y.2...@wcm-solutions.de wrote: Hi! Hanno Schlichting wrote: Stephan Richter has volunteered to do SVN to Github conversions for all Zope projects and has already completed all of Zope 2 core and some actively used projects like five.localsitemanager. Does anyone have objections if I ask him to convert the CMF packages? Yes. I have objections. I'd like to keep contributing to CMF. But I'm not going to support GitHub Inc. by using its services Seriously? You do realise it's: a) free (for us) b) decentralised I appreciate you may have to do the occasional 'push' to a .com domain, but you've got to be kidding... Martin ___ Zope-CMF maillist - Zope-CMF@zope.org https://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-dev] AccessControl bug fixed
On 22 August 2012 18:30, Yusei TAHARA yu...@domen.cx wrote: Hello, I found a bug in ZopeSecurityPolicy and fixed it. http://svn.zope.org/AccessControl/trunk/src/AccessControl/ZopeSecurityPolicy.py?rev=127548r1=113657r2=127548 Is it possible to release new version? Are we sure this wasn't done on purpose? At least it needs some review, there's lots of weird caching and lazy loading of global variables in that module. I *think* it's fine looking at the diff, but a second opinion would be useful. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [Checkins] SVN: zc.buildout/ Moved to github
On 20 August 2012 01:44, Ross Patterson m...@rpatterson.net wrote: For me the discussion sounds a little like a general denial against github using the legal story as rationale. +10. I'm so glad others are saying the things I think need saying. I *am* a signed ZF contributor and from experience, the likelihood of such stop energy or other unpleasantness prevents me from contributing to Zope projects nearly as much as I'd like or could. This is a sterling example. To be clear, I'm not invalidating legal concerns, I'm only frustrated that those representing those concerns are taking a hard line on only one concern without seeming to accept multiple invitations to work the problem from all represented concerns. I'm grateful to the others for trying so hard to kickstart a healthy level of participation in zc.buildout development once again. It is mildly interesting to compare the volume of discussion about Zope development vs the volume of discussion about where not to do Zope development on this list in the last few days. I think Jens is right to point out the legal concerns, which many of us don't fully understand. I think it might have been more effective had it pointed out why people should care, rather than just saying this is the rule. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] using WSGIPublisher
On 7 June 2012 07:20, Michael Howitz m...@gocept.com wrote: Am 06.06.2012 um 19:58 schrieb Hanno Schlichting: […] As I said above, my main concern is keeping publisher events and exception views intact. Some of these events need to happen in code that's currently inside repoze.* middleware. Like before transaction commit, publication failure or publication success. […] +1 to re-add these events. In our WSGI projects it hurts that they are gone now. There was a thread on this a while ago, and I did some in-depth research on the current state of WSGI publishers. See http://old.nabble.com/Zope-2-WSGI-investigation-to33063118.html#a33063118. Summary: use infrae.wsgi for now. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] using WSGIPublisher
On 7 June 2012 07:20, Michael Howitz m...@gocept.com wrote: Am 06.06.2012 um 19:58 schrieb Hanno Schlichting: […] As I said above, my main concern is keeping publisher events and exception views intact. Some of these events need to happen in code that's currently inside repoze.* middleware. Like before transaction commit, publication failure or publication success. […] +1 to re-add these events. In our WSGI projects it hurts that they are gone now. There was a thread on this a while ago, and I did some in-depth research on the current state of WSGI publishers. See http://old.nabble.com/Zope-2-WSGI-investigation-to33063118.html#a33063118. Summary: use infrae.wsgi for now. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding broken/missing support to zope.interface? (was: experimental.broken - Graceful handling of broken interfaces and components in the ZODB)
On 9 April 2012 15:41, Brian Sutherland br...@vanguardistas.net wrote: On Sun, Apr 08, 2012 at 01:04:37PM -0700, Ross Patterson wrote: experimental.broken is working well for me. It greatly aided me in getting through a particularly nasty upgrade allowing me to cleanup the ZCA cruft left by poorly behaved add-ons. I'd like to proceed with adding the core of this to zope.interface and I need the developers/maintainers to weigh in. The first and most fundamental matter is changing interface pickles such that they can be unpickled into something that can fulfill the minimum interface contract and don't break the ZCA. To that end, I'd like to add the following to zope.interface.interface: ... try: from ZODB.broken import find_global from ZODB.broken import IBroken def find_interface(modulename, globalname, Broken=IBroken, type=InterfaceClass): Find a global interface, returning a broken interface if it can't be found. return find_global(modulename, globalname, Broken=IBroken, type=InterfaceClass) except ImportError: IBroken = Interface def find_interface(modulename, globalname, Broken=IBroken, type=InterfaceClass): Find a global interface, raising ImportError if it can't be found. # From pickle.Unpickler.find_class __import__(module) mod = sys.modules[module] klass = getattr(mod, name) return klass ... class InterfaceClass(Element, InterfaceBase, Specification): ... def __reduce__(self): if self is IBroken: return self.__name__ return (find_interface, (modulename, globalname)) ... -1 For a number of reasons, but mainly because you add a test dependency on the ZODB from zope.interface. I think that zope.interface is such a fundamental package and the dependency is unacceptable. It's a soft dependency only, looking at the code sample. There has lately been a LOT of work done reducing the dependency structure of zope.* packages. You need a VERY good reason to start reversing that. It doesn't add any more (required) dependencies. This is a real issue that is breaking the sites of a lot of real users of zope.interface in a way that is hard to debug and reverse. Can you think of a better way to get around it? Other than don't get into the situation which isn't a valid answer as long as the ZTK ecosystem supports persistent local components. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2 WSGI investigation
On 3 January 2012 08:01, Chris McDonough chr...@plope.com wrote: Am I right in thinking Pyramid no longer uses repoze.tm2 or a middleware approach? What was the rationale for that design decision? You're right, Pyramid scaffolding no longer supplies repoze.tm2 or any other WSGI middleware component for handling transaction commits/aborts. Instead, it uses a Pyramid tween, which is sort of like internal Pyramid middleware inasmuch as its a user-manageable functional composition terminating at the application. See http://www.slideshare.net/aconrad/alex-conrad-pyramid-tweens-ploneconf-2011 Tweens can be reordeded arbitrarily, and the Pyramid exception handling logic (which locates and executes a Pyramid view when an application exception is raised) is itself a tween. The rationale is that application-specific error logic often needs to be executed after the transaction has been committed or aborted due to an exception. For example, if an exception is raised by an application, you'd like the system to abort the transaction, then potentially display a custom internal server error page (e.g. twitter fail-whale). People want to be able to write the error logic within their existing Pyramid development environment (where they have access to templating systems, a real request object, and possibly their database and other niceties) instead of as WSGI middleware higher in the stack than the transaction manager. I mostly agree with this, and for this reason I think the ZPublisher.Pubevents (not quite as good as tweens, I admit) and exception views are useful. With a middleware solution, you may find yourself passing application state in the wsgi environ which seems wrong. FWIW, I believe Tres still disagrees strongly with this decision. I can certainly see the use case for distributing transactions across middleware and/or composite apps, which is why it may make sense for this to be opt-in. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2 WSGI investigation
On 2 January 2012 08:50, Wichert Akkerman wich...@wiggy.net wrote: On 01/01/2012 08:39 PM, Martin Aspeli wrote: Hi, There are three known WSGI implementations of the Zope 2 publisher. I've had a look at them and made some notes about what I think provides the best story: ## Zope 2.13 WSGIPublisher Pros: * Allows distributed transaction management with repoze.tm2 * Allows distributed retry with repoze.retry * Ships with Zope * Quite simple Cons: * Requires repoze.tm2 and repoze.rety Why is that a con? I use repoze.tm2 and repoze.retry with all my pyramid projects and they work beautifully. Only insofar as it means you have to have these in your WSGI pipeline or it won't work, so there are more places things can go wrong. You'll note that I also consider not supporting such things a con for infrae.wsgi. I wouldn't get too hung up on it, I was mainly just trying to bring out the differences. It'd be nice if it wasn't a hard requirement, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 2 January 2012 12:33, Jens Vagelpohl j...@dataflake.org wrote: Hi Martin, Sphinx on svn.zope.org works for me. :) I have created a simple buildout and put it in SVN: http://svn.zope.org/zope_secrets/ The output is shown at http://docs.zope.org/zope_secrets/ and linked from the front page at http://docs.zope.org/. Every 6 hours, a cron job looks to see if the SVN revision has changed and if it has then the output is regenerated. Thanks! Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2 WSGI investigation
On 3 January 2012 06:39, Chris McDonough chr...@plope.com wrote: On Mon, 2012-01-02 at 10:39 +, Martin Aspeli wrote: On 2 January 2012 08:50, Wichert Akkerman wich...@wiggy.net wrote: On 01/01/2012 08:39 PM, Martin Aspeli wrote: Hi, There are three known WSGI implementations of the Zope 2 publisher. I've had a look at them and made some notes about what I think provides the best story: ## Zope 2.13 WSGIPublisher Pros: * Allows distributed transaction management with repoze.tm2 * Allows distributed retry with repoze.retry * Ships with Zope * Quite simple Cons: * Requires repoze.tm2 and repoze.rety Why is that a con? I use repoze.tm2 and repoze.retry with all my pyramid projects and they work beautifully. Only insofar as it means you have to have these in your WSGI pipeline or it won't work, so there are more places things can go wrong. You'll note that I also consider not supporting such things a con for infrae.wsgi. I wouldn't get too hung up on it, I was mainly just trying to bring out the differences. It'd be nice if it wasn't a hard requirement, though. FWIW, when I wrote repoze.tm2 I did not know that the transaction module already supported retry, so at least repoze.rety should die in favor of logic in something-like-repoze.tm2 that looks a little more like this: https://github.com/Pylons/pyramid_tm/blob/master/pyramid_tm/__init__.py#L49 (that's obviously not WSGI middleware, I'm just showing what the general logic should look like) Am I right in thinking Pyramid no longer uses repoze.tm2 or a middleware approach? What was the rationale for that design decision? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 09:44, Jens Vagelpohl j...@dataflake.org wrote: On Dec 31, 2011, at 20:09 , Martin Aspeli wrote: Hi folks, I have documented some of the darker corners of Zope's internals. I put it in the Plone developer documentation for lack of a better place, but it's not Plone-specific: http://collective-docs.readthedocs.org/en/latest/zope_secrets/index.html Hi Martin, There *is* a better place, docs.zope.org. If you can tell me where the sources are I can put it there. Sure: Clone https://github.com/collective/collective.developermanual/ and get it from source/zope_secrets. Where is docs.zope.org maintained? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 10:43, Jens Vagelpohl j...@dataflake.org wrote: Hi Martin, There *is* a better place, docs.zope.org. If you can tell me where the sources are I can put it there. Sure: Clone https://github.com/collective/collective.developermanual/ and get it from source/zope_secrets. Thanks, I'll take a look at it today. Where is docs.zope.org maintained? On one of the ZF servers. If it's going to go there, I'd like it to (a) be in version control and (b) be somewhere that I can edit it. Is that doable? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 10:51, Andreas Jung li...@zopyx.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Aspeli wrote: On 1 January 2012 10:43, Jens Vagelpohl j...@dataflake.org wrote: Hi Martin, There *is* a better place, docs.zope.org. If you can tell me where the sources are I can put it there. Sure: Clone https://github.com/collective/collective.developermanual/ and get it from source/zope_secrets. Thanks, I'll take a look at it today. Where is docs.zope.org maintained? On one of the ZF servers. If it's going to go there, I'd like it to (a) be in version control and (b) be somewhere that I can edit it. Is that doable? Wouldn't it make sense to integrate your docs with The Zope Book. It's maintained using Sphinx and the sources are on svn.zope.org (somewhere). This is really low level documentation. The Zope Book is for people using Zope. This is for people who may need to maintain or deep-debug it. I'm happy for it to be integrated if people think it makes sense, but I think it may be quite off-putting to read what is in many places block-by-block explanations of what the code does. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 11:00, Jens Vagelpohl j...@dataflake.org wrote: On Jan 1, 2012, at 11:46 , Martin Aspeli wrote: Where is docs.zope.org maintained? On one of the ZF servers. If it's going to go there, I'd like it to (a) be in version control and (b) be somewhere that I can edit it. Is that doable? That's how we do it with almost everything underneath the docs.zope.org hostname. The sources are on svn.zope.org are are pulled/built regularly. Where the source comes from doesn't really matter. The only requirement is that it should be a scriptable buildout process, like a buildout/Sphinx setup. Sphinx on svn.zope.org works for me. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope 2 WSGI investigation
Hi, There are three known WSGI implementations of the Zope 2 publisher. I've had a look at them and made some notes about what I think provides the best story: ## Zope 2.13 WSGIPublisher Pros: * Allows distributed transaction management with repoze.tm2 * Allows distributed retry with repoze.retry * Ships with Zope * Quite simple Cons: * Requires repoze.tm2 and repoze.rety * Does not properly emit publication events - possible patch in https://gist.github.com/1548061 * Does not do error handling or exception views * Claims not to properly implement streaming (though there is some code for it) * Probably less well tested than infrae.wsgi and repoze.zope2 (at least there is zero documentation) ## infrae.wsgi Pros: * Clean and well documented * Properly emits publication events * Supports streaming * Supports simplified virtual hosting with X-VHM-Host * Supports exception handling / error views * Reportedly has significant production use Cons: * Not 100% compatible (but close and fixable) - fix to make plone.transformchain work is here: https://gist.github.com/1547328 * Unnecessary five.grok dependency (but easy to rewrite to use ZCML registration) * No support for middleware transaction and retry management, so these can't be distributed across a WSGI pipeline * Error logging will not support ZMI error_log and assumes single process * Error handling is slightly different to standard publisher's exception views, and also does not honour existing standard_error_message etc ## repoze.zope2 Pros: * Clean and well documented * Reimplements and simplifies the BaseRequest.traverse() code, with comments * Supports distributed transaction management and retry Cons: * Replicates a lot of Zope startup code * Has now-unnecessary code to manage instances and configuration * repoze.obob abstraction is unnecessary since nothing else uses this * Does not emit publication events - possible patch in http://bugs.repoze.org/issue181 * Does not do error handling or exception views * Problems with file resources (does not properly traverse to browserDefault() result) -- possible patch in http://bugs.repoze.org/issue64 * Requires various middleware (repoze.tm, repoze.retry, repoze.vhm) ## Suggested approach going forward * Integrate infrae.wsgi into Zope 2 * Remove its five.grok dependency * Use the same exception-views protocol as ZPublisher (mainly, that the view name is ``index.html``) * Stop using __ 'private' variables in response.py to make it easier to work with * Add some BBB support for existing error logging and error messages Thoughts? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope internals documentation
Hi folks, I have documented some of the darker corners of Zope's internals. I put it in the Plone developer documentation for lack of a better place, but it's not Plone-specific: http://collective-docs.readthedocs.org/en/latest/zope_secrets/index.html Topics covered include startup, publication, traversal and security. One reason to do this, apart from morbid fascination, is to provide a baseline against which we can consider simplifying some of this stuff. For example, I'd like to consider an (opt-in) simplification of the publisher and traversal, probably based on a stripped-down and modernised repoze.zope2, which does away with some hooks and edge cases, but is much simpler and easier to understand. Some things we could consider chopping are: - Attribute traversal to anything other than methods at the end of the traversal chain (i.e. use __getitem__ traversal only) - Traversal to anything without explicit security declarations - The docstring security check - Maybe __bobo_traverse__ (i.e. just implement __getitem__) and __before_publishing_traverse__ (use a BeforeTraverseEvent instead, and notify this for all traversals, not just over local component sites) - All differences between publication and path traversal This is still somewhat half-baked and obviously would break things and require at least a new major version of Zope, but I think it's worth exploring at least. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] PAS and AccessControl bug?
Hi, I found this code in PAS, which is mostly lifted from AccessControl.userfolder: def _getObjectContext( self, v, request ): request - ( a, c, n, v ) o 'a 'is the object the object was accessed through o 'c 'is the physical container of the object o 'n 'is the name used to access the object o 'v' is the object (value) we're validating access to o XXX: Lifted from AccessControl.User.BasicUserFolder._getobcontext if len( request.steps ) == 0: # someone deleted root index_html request[ 'RESPONSE' ].notFoundError( 'no default view (root default view was probably deleted)' ) root = request[ 'PARENTS' ][ -1 ] request_container = aq_parent( root ) n = request.steps[ -1 ] # default to accessed and container as v.aq_parent a = c = request[ 'PARENTS' ][ 0 ] # try to find actual container inner = aq_inner( v ) innerparent = aq_parent( inner ) if innerparent is not None: # this is not a method, we needn't treat it specially c = innerparent elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = aq_inner( v ) # if pub's aq_parent or container is the request container, it # means pub was accessed from the root if a is request_container: a = root if c is request_container: c = root return a, c, n, v Look at this bit again: elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = aq_inner( v ) In AccessControl, it's similar: elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = getattr(v, 'aq_inner', v) Surely, this isn't right? What is the correct thing to do here? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZopeSecurityPolicy global manipulation
Hi, AccessControl.ZopeSecurityPolicy contains this code: from types import MethodType # AccessControl.Implementation inserts: # ZopeSecurityPolicy, getRoles, rolesForPermissionOn from AccessControl.SimpleObjectPolicies import _noroles rolesForPermissionOn = None # XXX: avoid import loop tuple_or_list = tuple, list def getRoles(container, name, value, default): global rolesForPermissionOn # XXX: avoid import loop if rolesForPermissionOn is None: from PermissionRole import rolesForPermissionOn roles = getattr(value, '__roles__', _noroles) if roles is _noroles: if not name or not isinstance(name, basestring): return default if type(value) is MethodType: container = value.im_self cls = getattr(container, '__class__', None) if cls is None: return default roles = getattr(cls, name+'__roles__', _noroles) if roles is _noroles: return default value = container if roles is None or isinstance(roles, tuple_or_list): return roles rolesForPermissionOn = getattr(roles, 'rolesForPermissionOn', None) if rolesForPermissionOn is not None: roles = rolesForPermissionOn(value) return roles Look carefully at how ``rolesForPermissionOn`` is used both at the top, to lazily set a global to avoid an import loop, and the bottom, as an attribute of the ``roles`` object. I'm pretty sure this is wrong™ on many levels, but most importantly, it seems the global is being overwritten each time execution gets down to that last block. I know this module gets munged by Implementation, but I'm pretty sure ImplPython doesn't define getRoles() at least, and I'm not even sure the C implementation does either. To prove it to myself, I made a frivolous equivalent that used 'datetime.date' as the importable. It's a bit ugly, but you get the idea: date = None class C(object): ... def __init__(self, d): ... self.date = d ... c1 = C(lambda: 'x') c2 = C(lambda: 'y') def get(c): ... global date ... if date is None: ... from datetime import date ... date = getattr(c, 'date', None) ... if date is not None: ... print date() ... date is None True get(c1) x date function lambda at 0x10dac8140 get(c2) y date function lambda at 0x10dac8cf8 Surely, this is all evil volatile? Maybe the global bit just needs to go away? It doesn't seem to be used in that function, and I'm pretty sure the implementation ends up overwriting the global anyway. Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Enabling External Methods in skin folders
On 17 November 2011 11:28, li...@nidelven-it.no wrote: Hi, I have a bunch of External Methods I'd like to make available in a skin form, and which reload in the same way a page template would if it was modified and the server was in debug mode. External methods should not require restarts either. What's the recommended product for enabling this now? A more robust approach may be to turn your external methods into views, utility functions called from other views, portal_setup upgrade steps, or whatever other purpose they serve. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 release management
Hi, On 17 November 2011 12:25, Laurence Rowe l...@lrowe.co.uk wrote: Along with David Glick, I would like to volunteer for the Zope 4 release management role, where I would take responsibility for producing the initial release of Zope 4 and David would then take over for the maintenance releases. w00t :-) +1 from me In doing so, I thought it would be helpful to set out our understanding of the mission of Zope 4. If accepted I'll work to produce a first draft of a roadmap based on the outcome of the recent sprints and discussions on this mailing list. Mission --- Zope 4 will be the first of several releases that seek to simplify our software stack. Over the years Zope 2 grew through the adoption of new technologies, notably Zope 3, but rarely removed older ways of doing things. Below, 'Zope 4' refers to the series of releases including Zope 5, 6, etc. Over the series of releases, Zope 4 will progressively remove more and more software as we transition to using software components shared with other Python web frameworks. It is as important to state what Zope 4 *is not* as what it should be: * Zope 4 will not seek to be of interest to those developing new applications. They should instead look to projects such as Pyramid that build on the experience and technologies of Zope without regard for the backwards compatibility concerns that will constrain Zope 4. * Zope 4 will not seek to innovate in itself but encourage innovation in software components shared with the wider Python web community. * Zope 4 will not move so quickly that it becomes impossible to make Plone, its main consumer, work with it. * But neither will Zope 4 seek to retain backwards compatibility at any cost. As the basis of Plone 4, Zope 2.13 will see maintenance releases as long as Plone 4 is supported. * Zope 4 will not have a release cycle independent of Plone. Zope 4 only exists as a transitional path for Zope 2 based applications and experience has shown that Zope 2 releases not used in any Plone release do not receive the same level of ongoing maintenance. I think these are sensible principles, but we shouldn't disregard the Zope community outside Plone. Hence, if Zenoss, Silva, ERP5 and others are willing to contribute and maintain code, they should not feel shunted out of the development process. That's not to say we should succumb to indefinite maintenance of all components on the odd chance iet may be needed by someone (we'll have a stable Zope 2 branch for that), but I believe we're stronger as a bigger community with more voices than as a narrow group with only one goal. Development Process --- We want to encourage all features to be developed on separate feature branches so we can maintain a stable trunk. Before these feature branches are merged they should be posted to the mailing list for review. This process will necessitate a lot of merging, so I want to propose that we move to Git for development (something we found very helpful at our recent San Francisco Zope 4 sprint.) I don't have any opinion on where the canonical repository should be hosted - I know some people have strong opinions that this should be on Zope Foundation controlled hardware. If that's to be the case then we will need the svn.zope.org maintainers to setup a shared git repository on that machine. I think mirroring to github (and/or launchpad in future) will be the simplest way to provide an anonymously accessible and web browsable copy. +1 - GitHub is really useful. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 roadmap
On 17 November 2011 14:46, Laurence Rowe l...@lrowe.co.uk wrote: Here's my current understanding of the Zope 4 roadmap. Zope 4 -- Significant progress has already been made on the following features and I expect they should all land in time for a Zope 4 release: - Storing parent pointers (elro, davisagli): we have a branch of Zope that changes OFS to store parent pointers on objects implementing ILocation and fixed the issues that arose in copy/cut/paste and zexp export code. There's an issue remaining with Acquisition, where we lost some protection against cycle protection - Hanno continues working on this. See also: http://wiki.zope.org/zope2/SetParentAndNameInOFS +1 - Remove XML zexp export (davisagli). +1 - Catalog using intids (rossp): we have branches of Zope, ZCatalog and CMF which change the catalog to use intids as their internal uid and rid. This needs more testing, but looks very promising. Currently it uses plone.app.intid / five.intid to maintain an intid to physical path mapping. Once the parent pointer work is stable, we can stop doing this and load objects directly from the database connection +0 - can we articulate the benefit of doing this? - Chameleon (chaoflow): Florian worked on making Zope use Chameleon by default. This works for the most parts, but there's some problems left in tests, as Chameleon wants an utility to be registered but a lot of tests in Zope and CMF don't use ZCML test layers. +1 - WebOb (garbas): Rok worked on bringing in WebOb as an underpinning for the request/response objects and making both API's available at the same time. I think the request is done and a good deal of the response works. Doing this means we can deprecate parts of the old request API and encourage the use of the more standard WebOb API. +1, though we'll need to balance the desire to be a better WSGI citizen with adding yet more complexity to BaseRequest and BaseResponse. - WSGI (matthewwilkes, hannosch): We fixed the bugs that had been reported about using the current WSGI support (mod_wsgi problems, forced dependency on repoze.who) on trunk and the 2.13 branch. Afterwards Matthew continued on a branch to remove the ZServer/medusa from Zope and leave only the WSGI publisher in place. +1 - I think WSGI should be the only way to deploy Zope 4. - Decorators for AccessControl (chaoflow). +1 Future releases (Zope 5, 6, etc.) - There has been some discussion on these topics but not much in the way of code yet: - Traversal Simplification. Remove attribute lookup from default traversal. +1, although this will require a lot of fixes in Zope consumers. I think we need a substantially simpler traversal mechanism that people actually understand. I've pdb'd BaseRequest.traverse more times than I care to remember and I still only vaguely understand it. - Unicode IDs in OFS +1 - Remove __allow_access_to_unprotected_subobjects__=1 from OFS.SimpleItem.Item +1, though again will break quite a lot. It's scary from a security perspective, though. - New ZMI +0 - we certainly need better XSRF protection and accessibility and look and feel are not great, though I think we should also consider what we actually use the ZMI for. A low-level object browser is really useful as a last resort admin tool. A number of the other things (like the various CMF tools, PAS, etc) could just as well expose their own views more independently of the ZMI, though we'd still need a way to discover those views. Perhaps something more simliar to the Plone control panel where tools can register views that just appear in a list of things you may need to manage may be useful, but it'll need some way of rooting that to e.g. Plone sites. - Move authentication out to WSGI middleware. +1 - anything we can do to make AccessControl simpler and more debuggable would be a big win. Long term - - Move to WebOb as our request object. The wider Python web framework community seems to have settled on WebOb as their request object of choice, so to facilitate sharing of software components with other projects we should consider making WebOb our request object too (instead of just wrapping it). This will require buy-in from the wider ZTK community as the ZTK components will need +1 - Move to Python 3. +0 Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 release management
On 17 November 2011 15:50, Jim Fulton j...@zope.com wrote: On Thu, Nov 17, 2011 at 7:25 AM, Laurence Rowe l...@lrowe.co.uk wrote: ... (Interesting roadmap snipped) This process will necessitate a lot of merging, so I want to propose that we move to Git for development (something we found very helpful at our recent San Francisco Zope 4 sprint.) I don't have any opinion on where the canonical repository should be hosted - I know some people have strong opinions that this should be on Zope Foundation controlled hardware. If that's to be the case then we will need the svn.zope.org maintainers to setup a shared git repository on that machine. Why on that machine? Why not have the ZF set up git.zope.org? As the primary maintainer of svn.zope.org, I'm not volunteering to have more stuff there. :) I think mirroring to github (and/or launchpad in future) will be the simplest way to provide an anonymously accessible and web browsable copy. I haven't used GitHub myself, but I gather it's good. :) Why not just let them host the project? That's the conclusion Plone came to. We have experience of running such a migration now, so can probably help. We also have some mirroring for backup happening. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 release management
On 17 November 2011 16:32, Tres Seaver tsea...@palladion.com wrote: * Zope 4 will not seek to innovate in itself but encourage innovation in software components shared with the wider Python web community. I smell something funny in here: if we aren't innovating, why are we making the effort? The innovation is in making it possible for users of Zope 2 to better be part of the wide Python web framework community and use tools and frameworks that are also in use in frameworks like Pylons/Pyramid, Django and TurboGears. The other innovation is in making our platform leaner, more maintainable and easier to understand and debug. * Zope 4 will not move so quickly that it becomes impossible to make Plone, its main consumer, work with it. We should be working to enable the other Zope2-consuming projects to keep up, too. +1 * But neither will Zope 4 seek to retain backwards compatibility at any cost. As the basis of Plone 4, Zope 2.13 will see maintenance releases as long as Plone 4 is supported. As long as any significant body of developers commits to maintaining it, even if the Plone devs don't care any more. +1 * Zope 4 will not have a release cycle independent of Plone. Zope 4 only exists as a transitional path for Zope 2 based applications and experience has shown that Zope 2 releases not used in any Plone release do not receive the same level of ongoing maintenance. I would actually argue that Zope4 have no real release cycle at all: instead, the individual pieces which make up Zope should have their own cycles, with perhaps a ZTK-like tracking set that Plone and others use as platform targets. -1 - we'll need something to amalgamate this into a release and we'll need a way to manage and number those releases. We want to encourage all features to be developed on separate feature branches so we can maintain a stable trunk. Before these feature branches are merged they should be posted to the mailing list for review. This process will necessitate a lot of merging, so I want to propose that we move to Git for development (something we found very helpful at our recent San Francisco Zope 4 sprint.) Note that this question is *not* suitable for loudest voice on zope-dev wins ressolution. The software belongs to the Zope Foundation, which will make any such decision. The work done on github by the Zope4 sprinters in SF should be seen as scratchpads for work which will be migrated back into the canonical repository for each project. A couple of points for consideration: - - bzr and hg are pretty much isomorphic to git WRT this kind of practice. Both have claims on our community which Git does not (hg because it is the tool of choice for Python, bzr because we already use Launchpad). Note that I use Git daily, and the others somewhat less frequently: I'm not speaking from ignorance here. - - Merging feature branches in SVN is not *that* difficult, typically: I've done scores of such merges myself with almost no pain (and the really painful ones would have been pretty much as bad with git / bzr / hg). In the Plone community, we held a poll. GitHub won hands-down. The second choice was staying with self-hosted SVN GitHub is a big leap forward in software project support. It's also rapidly becoming a de-facto place for many people to look for software. We win if the people we want to encourage to fix bugs or contribute features have a low barrier to entry. Note that this also includes Plone developers working on Plone at this stage, since Plone has now moved to GitHub. So, my personal vote would be for Zope to use GitHub (with a backup mirror), allowing me to have a more integrated toolchain. Personally, I find GitHub substantially better than BitBucket, especially for collaboration, and Launchpad nearly unusable. I'd encourage you to look at usage and growth statistics for the three main hosting/collaboration services. I don't have any opinion on where the canonical repository should be hosted - I know some people have strong opinions that this should be on Zope Foundation controlled hardware. I would note that hosting Git repositories without Github reduces the value proposition substantially: Git's wins in merging are much less significant than the collaboration workflow changes which github makes possible (tracking pull requests, in particular). Launchpad provides a similar mechanism, albeit one which is less sexy to use. OTOH, github's bug tracker is nothing to write home about, compared to Launchpad. Right - Plone has chosen to stick with self-hosted Trac for bug tracking. I'd imagine Lanchpad remaining Zope's bug tracker. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce
Re: [Zope-dev] Revert removal of ++skin++ in Zope4?
On 16 November 2011 11:30, Christian Theune c...@gocept.com wrote: Going down into the new ZMI project I find it to be the most light-weight approach without adding an extra dependency. What is this project? ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-CMF] [dev] CatalogTool as utility
On 19 September 2011 14:56, yuppie y.2...@wcm-solutions.de wrote: Hi! Hanno Schlichting wrote: On Mon, Sep 19, 2011 at 11:57 AM, yuppiey.2011-E2EsyBC0hj3+aS/ vkh9...@public.gmane.org wrote: Currently CMF trunk contains some hacks to work around the catalog brain issues. But I hope there is a better solution. Maybe the ICatalogBrain methods getURL, _unrestrictedGetObject and getObject should have a REQUEST argument that is used instead of self.REQUEST? Any kind of feedback and help is welcome. Mmh, why don't we just use zope.globalrequest in ZCatalog directly? And create a new ZCatalog 2.14 release series with this. Then we don't have to wait for Zope 4.0 to include it. Using an explicit argument is always cleaner than using zope.globalrequest. And getObject() already has a (currently unused) REQUEST argument. And we might be able to provide a migration path for the API change: If we don't use registerToolInterface, we don't have to change getObject/getURL calls in places where we still use getToolByName. But with zope.globalrequest we can avoid modifying the API. So if it is fine to smuggle a zope.globalrequest dependency in Zope 2.13, that might be the better solution. Or did you mean to use ZCatalog 2.14 only in CMF? getURL() is an extremely common operation, and is often called in TALES expressions. -100 on making it take a mandatory request parameter when there are other solutions available. Martin ___ Zope-CMF maillist - Zope-CMF@zope.org https://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-dev] RFC: Proposal for merging jbohman-zope.registry branch of zope.component
Hi, On 17 August 2011 03:50, Tres Seaver tsea...@palladion.com wrote: - - Land 'zope.registry' as a full ZTK package, with its own Launchpad artifacts, etc. This step may also involve moving bugs from zope.component to zope.registry. This is not a major issue, but just be aware that there's a widely-used package plone.registry (which, in fact, has no dependencies beyond the ZTK) that serves a rather different purpose (http://pypi.python.org/pypi/plone.registry). It may be a bit confusing to people if we have a zope.registry that means something else, so perhaps we could call it something else? As I said, not a major concern. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] www.zope.org relaunched
On 7 July 2011 12:58, Andreas Jung li...@zopyx.com wrote: Dear Zope Community, on behalf of the Zope Foundation I please to announce the relaunch of the new www.zope.org web site. http://www.zope.org The old zope.org site will available for the time being in (reduced form) under http://old.zope.org Many thanks to my team: - Kai Mertens - Michael Haubenwaller - Jens Vagelpohl - Johannes Raggam Andreas Jung Zope Foundation Fantastic work - thanks so much for succeeding where so many (myself included) have previously failed. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 09:42, Hanno Schlichting ha...@hannosch.eu wrote: What you are describing is exactly what I meant by old legacy Zope2 applications. You should be able to use this style of development with Zope 2.13. But you won't be able to upgrade to newer versions of Zope 2 anymore and expect your code to work unmodified. We discouraged this style of development for the past six years. It should come as no surprise that it will come to an end at some point. I would've thought it would also be possible for those who rely on this to maintain the relevant eggs as optional installations against Zope 2.x, no? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 10:18, Hanno Schlichting ha...@hannosch.eu wrote: On Tue, Jul 5, 2011 at 11:03 AM, Martin Aspeli optilude+li...@gmail.com wrote: I would've thought it would also be possible for those who rely on this to maintain the relevant eggs as optional installations against Zope 2.x, no? The ZMI is not a package - we don't have a split into zope and zope.app in Zope2. Once there's no more ZMI, Products.PageTemplates stops using RestrictedPython and the OFS base classes don't inherit from Acquisition.Implicit anymore, it'll be really hard to keep the legacy development approach working. I think it might be useful to spell out the reasons behind this (here, or better yet, somewhere more permanent like zope.org). I can imagine people reading this and wondering why it's a good idea, especially those who have an investment in the existing technologies. Someone might try, but I think it's not a wise decision to spent any resources that way. At some point every application written in the legacy style has to be rewritten. I think it would be a better use of resources for anyone to start doing that than maintaining a dead-end. This is a pretty sweeping statement that I think could cause a lot of nervousness. It might be the right thing in many ways, but we need to at least provide a bit more context. If you're a business that's invested dozens of person-years into a product, the prospect of rewriting could seem fairly daunting. At least we, as the Zope 2 community, need to set out the case for change and some kind of idea of timing and transition path, even if that means in some cases getting to a long term maintenance release and in other cases evolving away from certain technologies whilst being confident to keep using others. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 10:31, Hanno Schlichting ha...@hannosch.eu wrote: On Mon, Jul 4, 2011 at 12:19 PM, yuppie y.2...@wcm-solutions.de wrote: Long-term maintenance for Zope 2.13 would give these projects/deployments at least a few more years. Yes. I'm willing to cut releases for it for quite a while. I just expect to see active maintenance from the Plone community to stop in a year or two. Judging from the ongoing maintenance we currently have for Zope 2.10 or 2.11 I don't think it's very realistic to expect much to happen once the Plone guys stop. What I'm outlining here has of course almost nothing to do with the original idea and scope of Zope 2. Maybe at some point this should get a different name ;-) I don't want to discuss names, but I think the next release from Zope trunk should be explicitly a new *major* release. I think that's perfectly fine. Since I broke backwards compatibility with a number of changes I did, a major version increase is warranted. So we just got ourselves a Zope2 version 3.0. And no, naming it 4.0 or 5.0 or anything else doesn't make it any better at all. So 3.0 is the most sensible one :) Boy, that's going to be confusing. :) I'd actually favour calling it Zope2 4.0 just to avoid any mix-up with the defunct Zope 3, although I don't think there are any particularly good options here. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 11:10, Jens Vagelpohl j...@dataflake.org wrote: On 7/5/11 11:56 , Martin Aspeli wrote: On 5 July 2011 10:31, Hanno Schlichtingha...@hannosch.eu wrote: So we just got ourselves a Zope2 version 3.0. And no, naming it 4.0 or 5.0 or anything else doesn't make it any better at all. So 3.0 is the most sensible one :) Boy, that's going to be confusing. :) I'd actually favour calling it Zope2 4.0 just to avoid any mix-up with the defunct Zope 3, although I don't think there are any particularly good options here. I actually think it's a brilliant idea to skip 3.0 and call it 4.0. As Martin said, the potential for confusion is very high. A 4.0 would not only steer around confusing Zope3 3.x and Zope2 3.x, it would also make it easier to move back to the simple Zope moniker without any qualifying number tacked on. People who only look at version numbers would now choose Zope 4.0 instead of falling into the unmaintained Zope3 trap. I would tend to agree, given that we now have Blue Bream. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
Hi, On 5 July 2011 11:26, Tobias Helfrich helfr...@know-it.net wrote: Hi Hanno On Tue, Jul 5, 2011 at 10:19 AM, Tobias Helfrich helfr...@know-it.net wrote: OK, so you do think that we might use Zope 2.12 for a quite long time without thinking about anymore updates? Will there be any security updates for Zope 2.12 in the future? You want to use Zope 2.13. 2.12 is at the end of its active maintenance cycle as is Python 2.6 (the only Python version it supports). Zope 2.13 brings support for Python 2.7, which is a long-term maintenance release. OK, thx for the info. So we will be able to use Zope 2.13 with the techniques mentioned before? That will give us another two years to think about going on with different styles. So basically Zope 2 will be the framework for Plone only, because the community which is/was using Zope 2 for standalone individual projects has decreased to nearly none. I think it would be very sad if that happened, especially since there evidently demand from other projects. What I think is clear is that to evolve Zope 2, we need to shed some baggage and make some deeper changes to allow us to achieve some of our goals (e.g. WSGI, simplified stack, simpler and more easily controllable security, less magical traversal, more comprehensible publisher). Plone is obviously a big consumer of Zope 2 and I would expect Plone to have a major influence on Zope 2's evolution. But ERP5 is another big consumer, and we shouldn't ignore that. So it might be a good idea to look for something completely different? I don't think that Plone will be able to do everything we want it to. Or do you think we can stick with Zope 2 but change the way of building applications to ZPT/TAL? So we have to get rid of all DTML and what about an alternative to the ZSQL Methods? I think keeping DTML as an optional installation should be quite feasible. It's just possibly not something that the core Zope 2 team want to maintain anymore in the standard distribution. ZSQL Methods may be a similar story, in fact, especially as they rely on DTML. However, I'd encourage you to look at SQLAlchemy, which is way nicer to work with. OK, i have subscribed to the mailing list today, so unfortunately i haven't found this sort of information anywhere else. I don't want to blame you or anyone else for that, but it's not nice to hear that right now :-( I think there is some characteristic bluntness in Hanno's emails, but please realise that none of this is going to happen over night, and existing codebases are not going to magically disappear. Sometimes we have to be a bit more radical to understand the art of the possible and build a future platform that will support future needs. That doesn't mean there can't be both migration paths and long-term stable versions. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
Something of a meta-comment on this thread: It sounds like people are broadly in agreement on the direction, but not communicating enough about what's actually going on. I think it would be useful to keep some kind of roadmap wiki on zope.org, or at least post to the list periodically saying, this is what we're about to start doing. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 3 July 2011 16:44, Hanno Schlichting ha...@hannosch.eu wrote: On Sun, Jul 3, 2011 at 7:09 AM, Chris McDonough chr...@plope.com wrote: Zope still needs to the virtual host monster (or something like it) even with the WSGI publisher; there's nothing equivalent in the WSGI world (unless you could repoze.vhm, which is essentially just the virtual host monster, and probably doesn't need to live in middleware; no one uses it except people who use repoze.zope2). I'm expecting us to use repoze.vhm. But I've left the VHM in the code, so it's easy to install one if you still need it. For some time I expect Plone to install a VHM as part of its installation process. I don't have any skin in this game, but FTR, Mike Bayer isn't feeling all that confident about Beaker's sessioning component (or so he has told me). Beaker was originally made as a caching component, and had sessioning jammed into it quite late; nobody is really maintaining the sessioning component of it now. Well, if I can choose between modern unmaintained code from Mike Bayer and stone-age unmaintained code from Zope, it's still an easy choice ;-) And looking at the basics of what Beaker does here, it's still much more useful and of better quality than what we have in Zope 2. If there's any other non-framework-specific session machinery out there, we could use that as well. But I think most other stuff is tied into Django. FWIW, we have a high-performance, high-load application in production on Plone 4 with collective.beaker relying heavily on sessions, and I'm not aware of any problems with it. We use the memcached backend across two physical servers and a large number of Zope clients. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Sharing session between different zope servers
Hi, You can use collective.beaker to manage your sessions with beaker, and store on the filesystem (if all on the same server) or memcached (if on different servers). That's a code change, though. Martin On 15 June 2011 17:50, Subish K S kssubish...@yahoo.com wrote: Hi, We have 10+ Zope (version 2.7) servers ( on different machines). Right now each Zope server independently handles session management which means that if a Zope server fails the users on that Zope server do not get redirected to a functioning Zope server but get stuck on the failed Zope server. Is there any way to manage the session of all the zopes servers in a common place. Thanks in advance... Regards, Subish ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] component registry navelgazing
On 12 June 2011 21:48, Chris McDonough chr...@plope.com wrote: Currently if you ask a registry to singly-adapt an object to an interface, and the object you're trying to adapt implements that interface, here's what happens: from zope.component.registry import Components c = Components() from zope.interface import Interface, implements class IFoo(Interface): pass ... class Foo(object): ... implements(IFoo) ... foo = Foo() c.queryAdapter(IFoo, foo) None In order to get the object itself back from such an adaptation, you need to use the default= argument. c.queryAdapter(IFoo, foo, default=foo) __main__.Foo object at 0x24a3910 This seems slightly inconsistent with the adaptation worldview imposed by getAdapter/queryAdapter. I think it would be more consistent if c.queryAdapter(IFoo, foo) returned foo if foo already implemented IFoo and there was no other more specific adapter registered for the IFoo/foo pair in the registry, no? +1 Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZEO Server service installation on Windows
Hi, I just found out that, since Zope 2.12 (or 2.13?) it's no longer possible to install a ZEO server as a Windows service. Previously, it used to be possible to do: bin\zeoservice install (where bin\zeoservice is installed by plone.recipe.zeoserver), but apparently no longer. Is this really the case? It seems a pretty serious omission in our Windows story, and all the more puzzling since zopectl (and the instance scripts installed by plone.recipe.zope2instance) has an install option that still works. Has anyone got a solution, or at least is willing to chase this down? Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation
Hi, On 4 April 2011 17:30, Wolfgang Schnerring w...@gocept.com wrote: So, how can we proceed here? Should I (and Thomas) try to get a proof-of-concept implementation of this based on plone.testing? Or should we think about what it takes to merge most of plone.testing's ZCA support into zope.component itself first? I think either approach is valuable, and not necessarily mutually exclusive. I do care about the plone.testing API, which is used in production, so bear that in mind. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts
On 28 March 2011 15:45, Tres Seaver tsea...@palladion.com wrote: The vast majority of the doctest testcases in zope.* packages fall into this category: poor isolation, lots of edge cases which would obscure any real narrative docs, of which there are almost none. I believe the conflict is intrinsic, here, and not just an accident of careless / naive implementation: exercising all the preconditions of the contract of the function-under-test makes for really poor documentation, but is essential to good testing. Agree. Here's what I've found and learned the hard way: doctests are sometimes easier / more fun to write the *first* time I write tests. It's easy to just start writing, and the narrative helps me think about what I'm doing. Plus, it feels like I'm saving time on writing docs. They are much worse all subsequent times. Maintenance becomes a soup. Quick tests for edge cases feel like they obscure the narrative, so I may forgo them. Refactoring is painful. Tool support is poorer, e.g. no stepping through with pdb and no pyflakes. And people find the docs underwhelming. If I'm doing it wrong, I'd like to see it done right. Manuel is kind of cool, but I'm not sure it really addresses the issue. It's a better doctest, not a better unittest. Most zope.component packages only get away with simple doctests because they are simple/small packages (which is a good thing, mainly, of course). One of the main objections to unittest(2) seems to be that it's Javaesque. Yes, JUnit was the first library to solidify many of the current, cross-language testing patterns. We shouldn't be so NIH-stricken that we can't see the benefit of being consistent with patterns used in most other languages and test frameworks these days. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi, On 27 March 2011 15:54, Uli Fouquet u...@gnufix.de wrote: The (limited) experiences with py.test, however, were awesome. Some points that are quite cool IMHO: - Easy finding of tests: just write some ``test_function`` in a ``test_module`` and it will be found and executed. That also makes py.test tests more readable and maybe more intuitive. I'm not sure this is always a good idea. It feels a bit implicit, and having a base class isn't really a big problem, IMHO. It seems a bit like the kind of thing that sounds cool (look, it's even easier!), but in practice makes little difference. - Lots of setup code (unrelated to fixtures) can simply be skipped. No need to do the ``testsuite = complex-testcase-collecting`` over and over again. Maybe the main point of py.test. You don't need that for zope.testrunner either, of course, at least not when using unittest base classes. - py.test is more widespread in the Python community (that's my impression; I can't proof it) What about nose? - Support of unittest/unittest2: you can write standard lib setups (defining TestCases; no need to also write testsuite-setup stuff) and they will be found/executed. zope.testrunner for instance does not support the new `setUpClass`/`tearDownClass` concept of unittest2 (yes, you would use layers in that case; but it might be nice if zope.testrunner would support also class-wide fixtures in unittest2-style; people from other worlds might expect that to work). zope.testing should definitely gain support for the new unittest2 hooks. That wouldn't be very hard, though. ;-) Main drawbacks I see on py.test side are: - Lack of layer support (yet). Maybe we can do something about that in `zope.pytest` based on `plone.testing.layer`. - Limited doctest support. It is quite difficult (AFAIK) to define fixtures for doctests or to even set the usual doctest options (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests are simply collected and executed and not much finetuning is possible. With zope.testrunner, you *do* need a test_suite method to run doctests. I think that's a good thing. Look at plone.testing's README for examples. My very own concern about the latter point is: if this cannot be solved satisfactorily (in terms of readability and ease of use at least), py.test will not become a really hot candidate in the testing framework game on the Zope side at all. There are too many doctests used already and whether you like them or not: every testing framework used should offer at least minimal support for doctest fixtures and doctest options. But I might just have missed these features in py.test and they are provided already. FWIW, I think we should stop using .txt doctests for unit tests. Doctests should be used to test *documentation* (the examples are valid). For actual unit tests, writing tests in a unittest class is almost always better in the long run. doctests don't scale well and discourage the kind of ad-hoc this seems broken, I'll just write a quick test or I just fixed a bug, better add a regression test testing. For now I think that there is absolutely no need to think about a general move to py.test for the ztk. I think there's benefit in unifying the concepts and support for concepts like layers so that people can use the test runner they prefer. Maritn ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hi Jim, On 25 March 2011 14:12, Jim Fulton j...@zope.com wrote: Agree. There is a problem in that provideAdapter() and friends don't use getSiteManager() - the always use the global site manager. And there are parts of zope.component that use module level variables directly, ignoring hooks. These are meant to work this way. If you want to do local configuration, you should explictly select the relevent registry/manager or call getSiteManager and use the result. I sortof understand, but it makes it impossible to let people use provideAdapter() co in test setup and still retain some kind of layered test setup, without the kind of hacks we do in plone.testing. -- but anyone could at any time call getSiteManager.sethook to change it! Seriously? Nobody calls that but deep infrastructure code. People do call zope.site.hooks.setHooks() sometimes, though, e.g. upon traversal. This was never meant to be an application-level feature. I find the notion that people would call these dureing traversal to be disturbing. Are you sure you're not confusing this with setSite? Sorry, I meant setSite() above yes. Although sometimes people call setHooks() and then setSite(site) in test setup, because setSite() doesn't work until setHooks() has been called once. I think this may sometimes just be cargo-cult, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation
Hi, On 26 March 2011 08:11, Wolfgang Schnerring w...@gocept.com wrote: Hello, * Martin Aspeli optilude+li...@gmail.com [2011-03-25 13:58]: plone.testing (which is Plone non-specific and will shortly be BSD licensed) allows for stacking of ZCA registries. [...] Again, plone.testing is the result of hours and hours of finding weird problems, so I'd recommend you don't discard the knowledge there. I think a best case scenario would be for plone.testing.zca to just be a delegate for something more robust in zope.component.testing. I wasn't aware plone.testing had something for the ZCA, before I read that in this thread yesterday. I'm glad to hear that, and I certainly don't want to discard anything. But from reading the code and what you wrote, I get the impression that plone.testing does not yet solve the issue completely, precisely because of the two points I brought up and that continue below: It certainly doesn't solve the unregister (non-?)use case. It has the advantage of being in use in the wild today, though. :-) On 25 March 2011 13:17, Jim Fulton j...@zope.com wrote: On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring w...@gocept.com wrote: we realized that just squeezing in a new registry and bending its bases to the previously active one is not enough for full isolation, since this does not cover *deleting* registrations Is deleting registrations important? This seems like an odd use case. If it's needed, I would suggest starting with a baseline (e.g. stack) that doesn't include the component you want to test deleting, then adding in setup. I've wanted to specifically nuke registrations sometimes, the pattern being, load the package's configure.zcml in the layer, and then delete something (to demonstrate some error handling behaviour). I agree that this use case is rare, but I'm not sure it is rare enough that we should ignore it. I need to think about this some more, though. It would be better to explicitly register what you need. I think you either need to consider a package's ZCML-loaded configuration as an atomic part of test setup, or you need to break it down into individual registrations. In the first case, your test fixture is package foo's configuration is loaded, which is of course valid. In the second case, your fixture is the following components, some of which happen to be in package foo, are registered. I don't think a fixture of package foo's configuration except component X and Y is all that useful. Of course, there may be cases where an unregister is needed, but this is probably something you should solve locally. For example, you can explicitly unregister the component at the beginning of your test and reinstate it at the end of your test, or indeed in layer setup and tear-down. Automating this with stacking is not a great win. Please also take my word for it when I say that copying the whole registry is non-trivial and would rely on brittle ZCA internals. I tried. :) 2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry. I think to provide test support for zope.component (i. e. generally, at the library level), we need to support both entry points. Why? Why would someone care about anything other than the current effective configuration. Because that's the API that zope.component offers, it conceptually deals with *two* registries: the one you get from getSiteManager and the one you get from getGlobalSiteManager. I agree that it is unlikely that client code will *read* registrations using getGlobalSiteManager -- since most everyone uses zope.component.getUtility etc. (which in turn uses getSiteManager). But *writing* is a different matter. Just one example: zope.component.provideUtility etc. uses getGlobalSiteManager, while the ZCML directives use getSiteManager. At least as long as zope.component.provide* uses getGlobalSiteManager instead of getSiteManager, I maintain that test infrastructure needs to 1. wrap the global registry 2. wrap whatever getSiteManager currently returns (Otherwise we might be able get away with not doing (1), but I think we'll always need to do (2).) plone.testing has it the other way around, doing (1) but not (2), as Martin says himself... We do definitely need to allow the global site manager to be stacked (which you can achieve with __bases__ as in plone.testing, unregistration notwithstanding). But once you do that, the rest is pretty easy. The local site manager will always have the global as one of its (nested) __bases__. My preference would be: - We replace the three separate module-level references to the global site manager with a single object - This object is a holder for a registry reference
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi, On 26 March 2011 14:18, Wolfgang Schnerring w...@gocept.com wrote: Because, while test layers are nice because they have the above properties, I'm not too happy with the current implementation, namely the use (or is it abuse? ;-) of __bases__ and __name__, which leads very naturally to not-so-helpful implementation patterns. As a concrete example, some of the test layers of ZTK packages, most notably the successors of zope.app.testing.functional in zope.component, zope.app.appsetup, zope.app.wsgi, are implemented right now in a way that does not have properties 2+3. The most straightforward way to improve that situation would be to use plone.testing, since that provides a layer implementation that is both sane and easy to use, and has all the nice properties. (Maybe even fold some of it into zope.testrunner itself.) I'd be happy to move layer.py from plone.testing to zope.testrunner for sure. But then I realized, that's a major undertaking, so we better think about what the actual goals are before changing lots of stuff. And then I heard people expressing interest in other test runners. (py.test and nose seem to be the biggest players, stdlib's unittest/unittest2 seems to be woefully behind in terms of test selection and other features like coverage or debugging. I for one happen to like zope.testrunner, but if there's a good alternative, why not combine our efforts?) I also found that it is rather hard to explain how to use test layers in a way so you actually get the above properties, so I started wondering whether there was a cleaner way to get them. Personally, I rely a lot on the layer mechanism and would not want to lose it. So I guess I have three questions: 1. (The starting point:) How can we improve the test infrastructure situation of the ZTK packages? I'd say having more reusable test layers would help. 2. (The main thing:) Are test layers, both the concept and the implementation as epitomized by plone.testing, the way to go or is there a cleaner alternative? When we looked at how to improve Plone's testing story, we couldn't find anything better. One thing I would like is a better way to support python setup.py test. 3. (But only tangentially, I don't want to sidetrack the discussion at hand nor start a flamewar:) Is there a compelling alternative to zope.testrunner that would make it worthwile to think about either generalizing the fixture support or even migrating away from zope.testrunner? It's a good question to ask, though I think this would be a really big undertaking for maybe only marginal gain (if any). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hi, On 25 March 2011 13:17, Jim Fulton j...@zope.com wrote: On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring w...@gocept.com wrote: Hello Uli, I've spent quite some time thinking (and partly coding) about the same issues you mention (but didn't feel ready to talk about it here, yet), so I'm glad that maybe we can start thinking about them together I think your email addresses two quite different topics, so I'll split my reply. First up: test support for zope.component. Second part: about the concept of test layers. * Uli Fouquet u...@gnufix.de [2011-03-24 01:05]: Right now we have a problem with pytest integration when it comes to ZCA setups [...] all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation. Exactly. This issue has bitten me too in various places, and as far as I know there are no solutions for it, yet. The classic solution is to start tests with empty registries, or, if you're using layers, with some baseline registries. plone.testing (which is Plone non-specific and will shortly be BSD licensed) allows for stacking of ZCA registries. It has to do some ugly hacking to achieve this, since zope.component stores handles to the global registry in *three* different modules and update them in weird ways depending on the registry hooking, but it's well tested and robust now. See http://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.py and http://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.txt. What I envision to solve this issue is that test support for zope.component should work the same way as with the ZODB. There, we have a *stack* of Databases (DemoStorages, to be precise) that wrap each other, where each one delegates reads downwards, but keeps writes for itself. So you might have one database for the layer that provides the baseline, and each test (in its setUp) gets its own database where it can do whatever it wants, because it is thrown away in its tearDown. In principle, quite a few of the mechanics to do the same thing with zope.component registries are already there (since a registry keeps a list of base registries it will delegate to when something can not be found in itself). And as Hanno and Godefroid mentioned, plone.testing does something in this direction already. (And, it bears repeating, in its core has no dependencies on Plone or Zope2.) I like the idea of stacking registries. plone.testing implements this. If we could fix zope.component to make the implementation less ugly, that'd be a big win. But as far as I see, there are issues that plone.testing does not address: 1. I've been going over this stuff with my colleague Thomas Lotze, and we realized that just squeezing in a new registry and bending its bases to the previously active one is not enough for full isolation, since this does not cover *deleting* registrations (one, you can only delete the registration from the precise registry it was created in, and two, in the just-bend-the-bases approach, once you delete a registration, it's gone forever). Correct, although this is in practice extremely rare. I'd say it's much better to control setup more carefully and not have to undo in a child layer. I think to provide full isolation, we need to make *copies*. And since zope.component in general supports a chain of registries, we probably need to make copies of each of them. Copying is very hard. It was my first attempt in plone.testing and didn't work out well. You need to support pickling of registries for local/persistent component registries. I cannot begin to tell you how many weird pickling errors I found and had to work around. Is deleting registrations important? This seems like an odd use case. If it's needed, I would suggest starting with a baseline (e.g. stack) that doesn't include the component you want to test deleting, then adding in setup. +1 2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry. I think to provide test support for zope.component (i. e. generally, at the library level), we need to support both entry points. Why? Why would someone care about anything other than the current effective configuration. Agree. There is a problem in that provideAdapter() and friends don't use getSiteManager() - the always use the global site manager. And there are parts of zope.component that use module level variables directly, ignoring hooks. The global one is not hard, but the getSiteManager one gets nasty really fast, because of course we have to rely on bending getSiteManager to return the current test registry But as you point
Re: [Zope-dev] Anyone want to do Google Summer of code mentoring for PSF?
Hi,
On 20 March 2011 15:00, Hanno Schlichting ha...@hannosch.eu wrote:
On Sun, Mar 20, 2011 at 3:28 PM, Jim Fulton j...@zope.com wrote:
- The mechanism shouldn't require something to grok/analyze the
code. The mechanism should be explicit. This is implied by
pythonic. I remember Grok being more implicit than skimming the
links above suggest. Perhaps Grok has has become more explicit than
I remember.
Both Grok and Pyramid (or martian and venusian really) do a scan of
the code to find the registration hints.
I think you cannot avoid this, if you want to support an explicit
configuration phase. Otherwise the first import of a module could
occur at any point at runtime and have a configuration side-effect
like registering a new view. Personally I find the venusian approach
pretty simple and explicit enough.
Note that the move from advice-based syntax to class decorators
provides a good opportunity to revisit how some of this works based
on experience using the ztk.
Taking one of the examples of grokcore.component, I think there's a
lot that can be made simpler:
import grokcore.component
from zope.i18n.interfaces import ITranslationDomain
class HelloWorldTranslationDomain(grokcore.component.GlobalUtility):
grokcore.component.implements(ITranslationDomain)
grokcore.component.name('helloworld')
Based on my Pyramid exposure, I'd write this as something like:
from something import utility
from zope.i18n.interfaces import ITranslationDomain
@utility(ITranslationDomain, name='helloworld')
class HelloWorldTranslationDomain(object):
pass
This does mean hardcoding default values into the registration calls.
I'm not sure I see the value of avoiding this, as long as there's a
way to unregister this utility and re-register the class with other
values.
I agree - this is nicer. In Grok's defence, class decorators didn't
exist when martian was conceived. :)
I do wonder whether it's nicer *though*, though, to justify yet more
proliferation of configuration syntax.
I appreciate that in Python 3 the in-class advice (which was pioneered
by zope.component/zope.interface, don't forget) may not work properly,
so we may not have any choice eventually. But saving two lines of code
in a hypothetical example doesn't necessarily outweigh the confusion
that comes from having multiple ways to do things, and grokcore.*
configuration is in use in the wild outside of Grok.
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Anyone want to do Google Summer of code mentoring for PSF?
Hi, On 20 March 2011 15:29, Jim Fulton j...@zope.com wrote: I think you cannot avoid this, if you want to support an explicit configuration phase. Otherwise the first import of a module could occur at any point at runtime and have a configuration side-effect like registering a new view. Personally I find the venusian approach pretty simple and explicit enough. I disagree. First, the notion that you'd import at run time is pretty odd, unless you count start-up in runtime. I think Hanno was saying that configuration at import time is unpredictable and leads to ordering problems and circular import type risks. This shouldn't really be a surprise to anyone. Second, well, really, I'm not ready to give up on a straightforward definition of wiring that doesn't rely on module scanning. I recall Chris suggesting some improvements of the zope.configuration API to make it user friendly. Right now, it only really makes sense in the context of writing ZCML directives. Pyramid has largely done that work, though I'm not sure how compatible it is at this point in time. With a cleaner, better documented zope.configuration API, you can do configuration in your __main__ function or whatever works on startup. One option then is to use the indirection of ZCML or the indirection of martian/venusian style scanning, which may make sense for frameworks and pluggable apps, but less sense for more closed systems. Then again, it feels like Pyramid has largely done this already. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-CMF] GenericSetup global registries
On 9 March 2011 14:09, Wichert Akkerman wich...@wiggy.net wrote: On 3/9/11 15:03 , Hanno Schlichting wrote: On Wed, Mar 9, 2011 at 3:00 PM, Wichert Akkermanwich...@wiggy.net wrote: You really want to use the ZCA for every occurance of a string-object mapping? Really? As long as Zope doesn't have a central configuration registry like Pyramid has, it unfortunately makes sense. It's massive overkill and we shouldn't need to force simple values into interfaces, but that's the current state of things. It worked fine without using the ZCA before, so I don't really see the problem that we're trying to solve here. It all looks like a workaround for a missing feature in plone.testing, which won't even help much since plone.testing needs to support older GS releases as well. I think that may be subjective. It worked fine before because no-one ever put more than one instance of the registry in the same process, and test isolation with ZopeTestCase/PloneTestCase is a joke, so people just accepted leaked state. Global registries maintained in global variables are an anti-pattern, precisely because they always make testing more difficult. We have a standard way to do registries (which may be 'overkill' for this use case in isolation, but not massively so, and as Hanno and Laurence pointed out, consistency of approach matters too). We should use it. plone.testing will evolve, and we'll have a policy whereby major version numbers align with Plone (and thus, indirectly, Zope) versions so as not to have to laden it with the type of BBB spaghetti that made PloneTestCase impossible to maintain. I'm all for cleanup. Next up, get rid of the super-insane global ZCA registry instance variables (yes, plural). But I digress. Martin ___ Zope-CMF maillist - Zope-CMF@zope.org https://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-dev] ZPublisher: using zope.formlib and z3c.form in Zope 2
Hi, On 2 March 2011 08:43, yuppie y.2...@wcm-solutions.de wrote: Hi! ZPublisher.Publish and zope.publisher.publish process form inputs differently. Zope 2 returns encoded strings unchanged if no converters are specified. zope.publisher converts encoded strings to unicode. One major reason why zope.formlib and z3c.form can't be used directly in Zope 2 is the fact they expect decoded form values. five.formlib uses special base classes and plone.z3cform monkey patches the base classes in z3c.form. Proposal: - Products.Five.browser.decode should be moved to ZPublisher. processInputs and setPageEncoding are publisher related code. +1 - After traversal and before calling the object ZPublisher.Publish.publish should figure out if the object expects zope.publisher behavior. Either we use a new interface for this or we use zope.publisher.interfaces.browser.IBrowserPage: AFAICS in Zope 2 land only zope.formlib and z3c.form based views implement IBrowserPage. Isn't this in zope.browserpage now? - If the object implements that interface, the request is post processed using processInputs and setPageEncoding. +1 - plone.z3cform uses a modified version of processInputs and doesn't use setPageEncoding. Can anybody explain why? I guess that are no z3c.form specific reasons. Maybe the changes can be merged back to Zope? processInputs() in Five was very buggy. I thought I'd merged those changes into Zope 2? I don't know what setPageEncoding() does, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] z3c.form buildout broken
On 21 November 2010 16:16, Roger d...@projekt01.ch wrote: Why do we use such crapy parts like omplette in z3c.form? I never do any development whatsoever without collective.recipe.omelette. It may not be right for z3c.form, but it's not crappy. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] lxml dependency in Zope 2.12.10 KGS
Hi, The Zope 2.12.10 KGS at http://download.zope.org/Zope2/index/2.12.10/versions.cfg specifies lxml = 2.2.6 There is no Python 2.6 Windows build for this egg, which means that this version cannot be installed on Windows under Python 2.6. Version 2.2.4 is the latest version with safe binary eggs for all platforms. What in Zope depends on lxml? Why did we pin to 2.2.6? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] lxml dependency in Zope 2.12.10 KGS
On 10 September 2010 14:04, Hanno Schlichting ha...@hannosch.eu wrote: Hi. On Fri, Sep 10, 2010 at 2:45 PM, Martin Aspeli optilude+li...@gmail.comoptilude%2bli...@gmail.com wrote: The Zope 2.12.10 KGS at http://download.zope.org/Zope2/index/2.12.10/versions.cfg specifies lxml = 2.2.6 There is no Python 2.6 Windows build for this egg, which means that this version cannot be installed on Windows under Python 2.6. Version 2.2.4 is the latest version with safe binary eggs for all platforms. This is unfortunate, but really a problem for the lxml community and not us. So the lxml community cannot keep up with providing binary Windows eggs. This cannot force us to stick with old and buggy versions of the software. Well... the problem, apparently, is that libxml2 doesn't (or didn't?) have suitable Windows binaries, or so I'm told. I'm also not sure the bug fixes in 2.2.5 onwards are very important in a Zope context, since they seem to deal with Python 3 compatibility mainly. What in Zope depends on lxml? Why did we pin to 2.2.6? 2.2.6 was the latest stable version available at the time of the release. 2.2.7 had known problems with newer libxml2 versions, but I see there's a 2.2.8 out now, which we should update to. I'm not sure about the actual dependency situation. I think it's more or less a convenience pin, as lxml is used very often in Zope related projects. We provide a known good version for it in the Zope Toolkit KGS for the same reason. This sounds wrong to me. If we *are* going to use a convenience pin, then surely the ability to install on the world's most-used operating system has to be part of the convenience. ;-) If we don't use it, we shouldn't pin it, IMHO. We found this problem because the Zope KGS was overriding another KGS where we had pinned lxml to 2.2.4. I don't think Zope has any business getting in the way of that. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] lxml dependency in Zope 2.12.10 KGS
On 10 September 2010 14:26, Hanno Schlichting ha...@hannosch.eu wrote: On Fri, Sep 10, 2010 at 3:17 PM, Martin Aspeli optilude+li...@gmail.comoptilude%2bli...@gmail.com wrote: If we *are* going to use a convenience pin, then surely the ability to install on the world's most-used operating system has to be part of the convenience. ;-) That's a lame argument. Windows is almost irrelevant for the market we are in - web server deployments. Erm, you think so? Maybe we should do a poll on how many Zope / Plone developers use Windows on the desktop. Or look at how many people download the Windows installer. You need a dev environment, not just deployment, and a lot of people are on Windows. Our own community is barely able to keep up providing the most basic Windows support and ensuring tests pass. As long as we don't have more community volunteers actually caring about Windows support, I won't let it be an argument to penalize the rest of the community. When the software breaks, people go elsewhere. I didn't say Windows support was easy, or any fun. But we have to decide: do we care about people who have made (or are forced to make) different technology choices than us, or do we tell them their platform is unsupported? If we don't use it, we shouldn't pin it, IMHO. We found this problem because the Zope KGS was overriding another KGS where we had pinned lxml to 2.2.4. I don't think Zope has any business getting in the way of that. The KGS is a base KGS you can use. Nobody forces you to stick to it. In fact for every single deployment of your own you will need to extend it. I don't see a problem with the few people using Windows and not installing compilers on their platforms to change one version pin. I think you're missing the point: - We shouldn't pin software we don't use. It may be well intentioned, but if we don't depend on it, we shouldn't take responsibility for it, or give the perception that we take that responsibility. - If we do depend on it, we need to make sure it works on the platforms we support. QA isn't something you do only when it's easy to do in your local dev sandbox. - If we suddenly no longer support Windows, we better have the guts to come out and say it, stop producing Windows eggs for Zope 2 stuff and explicitly state that people cannot and should not use Windows for Zope development. I hope that's not the case, though. ;) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Different Zope 3.4.2 Newbie feedback
On 9 September 2010 01:33, Christopher Lozinski lozin...@freerecruiting.com wrote: Here I am sharing my thoughts as a Zope 3.4.2 newbie. As you have been told three times this week already: Zope 3 is in effect dead. You want to look at Grok (if you want less ZCML and more convention-based configuration) and/or Blue Bream (which is basically the evolution of Zope 3.4 under a new, less confusing name). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZTK TTW Content Types Project Wiki
Don't be too harsh on Grok/Dexterity. Dexterity has worked out how not to repeat the definitions in interfaces, forms and content objects. It also produces an application with suprisingly little redundant code. I urge you to try it out. The benefits are of course quick turn around, version control, testability and the debugger. Also note that (the Plone integration for) Dexterity allows you to create types entirely through-the-web. This can then be meaningfully transitioned to filesystem development without forcing you to start from scratch. The big, big problem with any kind of TTW development is that it usually breaks down badly when you move your code to a production server and then need to continue development in a separate environment. Code and configuration stored in a database is difficult to deploy and merge when the same database holds live data and content. The same problem exists where more than one developer needs to collaborate. Without source files and source control, it's impossible to give each developer their own sandbox. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 3 Newbie Feedback
Hi, So how to make Blue Bream easier to use? I propose making the initial start up interface a simple content management system, kind of a Plone light. This seems like a bad idea. Blue Bream attempts to be a development framework, not a content management system. You are suggesting we make an apple look like an orange, but still taste like an apple. Perhaps someone will build a CMS on top of Blue Bream, and perhaps this will make BB easier for people to learn, but that would need to be a separate project with separate goals. The experience from Zope 2 and Plone is that muddying the waters between an application and a framework makes a system harder for people to understand and use, not easier. I know Plone just had a new release . I am not sure what all is in it, but it would be great if Zope 3 on startup looked just like Plone, as far as possible. Right out of the box allow users to register themselves, and add in the stock content types. Calendar item, news item, things like that, not the developer centric ZPT and DTML file. Also allow the developer stuff, but under a different branch of a hierarchical menu system.Maybe only people with a developer or manager permission see those kinds of items. It sounds like you're talking about through-the-web development, which is an idea now thoroughly discredited. TTW customisation has its place. Writing code through a web browser is just painful and brings long term pain once you grow out of tinkering mode and start having to worry about things like deployment and dev/test/prod lifecycles. That is all I want. A simple content management system out of the box, make it easy to add my own content type. Sure the serious hackers can throw all that out, but for the average end user/developer, that would be a hugely tempting improvement over using Plone. If you just want a simple CMS, I suggest you install a CMS. With Dexterity, by the way (available in Plone 4, probably standard in Plone 5), you can legitimately create content types through the web in Plone. If I can add a few dtml pages to display my content, it would all be very easy to get started. Later I could export to the file system, and svn. Please note that DTML is a dead (and horrid) technology. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] .lock files on Windows
Hi, With Plone 4 and thus Zope 2.12.10, we've noticed a problem that I think only affects Windows. Can anyone confirm or shed some more light? Basically, if we run an instance (installed via plone.recipe.zope2instance as bin\instance) in the foreground (bin\instance fg) and then kill it with Ctrl+C, the var\instance.lock and var\instance.pid files are not cleaned up. These need to be manually deleted, otherwise Zope refuses to start up with the confusing message Please stop the instance before starting it. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] .lock files on Windows
On 3 September 2010 17:06, Jim Pharis binbr...@gmail.com wrote: I'm on 2.13.0a3 w/plone.recipe.zope2instanec-4.0.2. Under the scenario Martin described, exiting a fg with ctrl-c, the lock file is cleaned up for me. If I kill the service using Task Manager the lock file remains. It seemed to be happening intermittently. But it happened on two different PCs running Windows XP. Instances were started with bin\instance fg and killed with Ctrl+C. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] (Zope2) a proposed tweak to assigning default roles to permissions
Hi, On 19 August 2010 16:46, Hanno Schlichting ha...@hannosch.eu wrote: Hi. On Thu, Aug 19, 2010 at 6:15 AM, David Glick davidgl...@groundwire.org wrote: As an alternative to requiring calling setDefaultRoles/addPermission at import time, I suggest that we add an optional roles attribute to the permission directive. This would then be used when the directive is executed, instead of the current hard-coded Manager setting. Examples: !-- a new permission with 2 default roles -- permission id=my.NewPermission title=My new permission roles=Manager SiteAdmin/ !-- a new permission with Manager as the sole, implicit role (backwards-compatible) -- permission id=my.OtherPermission title=My other permission/ Can roles currently contain whitespace? Like Awesome People? Yes, they can. If so, we should go for nested nodes: permission id=my.NewPermission title=My new permission roleManager/role roleSiteAdmin/role roleAwesome People/role /permission I think this matches the style of some of the GenericSetup handlers which deal with permissions. +1 Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: ETag support for zope.browserresource
On 10 August 2010 02:25, Marius Gedminas mar...@gedmin.as wrote: I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596 Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0. No strong objections, really, but bear in mind that people will likely want to customise this. With plone.caching / plone.app.caching we have a framework that, among other things, deals with browser resources and sets etag, last-modified and other headers according to rules configured by the developer and/or administrator. I think that would just stomp on etags set by zope.browserresource, but worth bearing in mind that for bigger applications like Plone, we need a centralised, overarching and configurable strategy for cache headers. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Changing and migrating persistence structure
Hi Jim,
On 08/08/2010, Jim Fulton j...@zope.com wrote:
On Thu, Aug 5, 2010 at 2:36 AM, Martin Aspeli optilude+li...@gmail.com
wrote:
...
I have a package (plone.registry) that currently has a persistent
structure like this:
Registry(Persistent)
|
+-- Records(Persistent)
|
+-- BTree of Record(Persistent)
|
+-- PersistentField(Persistent)
That is, a Registry is a persistent object containing a persistent
Records object that in turn contains a BTree of persistent Record
Since BTrees are mapping, I assume that you mean the values are
records and that the keys are something boring like strings or
integers.
Yes. The keys are strings.
I like to use mathematical notation when talking about BTrees and
sets, as in:
Registry
BTree {? - Record}
objects that contain a persistent PersistentField and a primitive
value.
This is quite inefficient, though, because it results in a lot of
object loads. On any given request, some of our projects load a dozen
or more values from the registry. Each is just a simple primitive, but
we need to load the full shebang to make it work.
Not sure what you mean by full shebang.
The Registry, Records object, the relevant Record in the relevant
BTree, and possibly the PersistentField object.
In the new branch it just looks up the value in the relevant BTree.
Now, I'd like to move to this structure:
Registry(Persistent)
|
+-- Records
|
+-- BTree of Field
|
+-- BTree of values
I'm foggy on what field and value are here or what your queries
are doing. Maybe this is just a distraction.
Somewhat, unless you've worked with plone.registry. The point is to
allow the get a value API to just look at self.values[key], which is
a fast lookup and doesn't load anything except the relevant BTree
bucket + the registry itself.
Here, there's only one Persistent object, plus the two BTrees: one
holding all the fields and one holding all the values. Records no
longer needs to be persistent (its attributes become part of the
parent Registry's _p_jar).
I wonder what role Records plays independent of the Registry.
None, really. The main reason to have it is to be able to have an API
like registry.records with dict-like notation (there's also
__getitem__ on the registry, which returns the value of a given key,
not the Record). I made ``records`` an attribute of type Records, and
Records derives from Persistent. I wish I hadn't, since it can just
live in its parent's _p_jar.
I also wonder why it matters whether it is persistent or not.
It's better if it isn't (one fewer object to load/fill up the cache),
though the real culprits are the many Record objects each being
persistent and loaded separately.
On a given request, we can end up loading a dozen or more values from
the registry, which means a dozen or more objects in the cache and
associated overhead.
Fields no longer need to be persistent
either, since they are in effect immutable objects. Values are
primitives anyway.
I've done this (in a branch) and it works for new sites. However, I'm
having a nightmare trying to migrate old sites. As soon as I access
anything that uses the registry, I get ZODB errors, because the
persistent structure is now different. In particular, it's trying to
read a value into e.g. a Records object that used to derive from
Persistent, but now no longer does.
What savings do you get by making Records non-persistent?
One fewer persistent object.
I think the real saving is in making the Record object non-persistent,
especially since the read use case can just read from the ``values``
BTree with the structure above.
What is the best way to manage this type of migration?
Today, it probably makes the most sense to make new classes for the
non-persistemnt objects. You'll then need to write a script to
rebuild the data structures.
Okay. So there's no way to get at the data if I take Persistent out of
the base classes for Records / Record.
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Changing and migrating persistence structure
On 8 August 2010 20:29, Hanno Schlichting ha...@hannosch.eu wrote: There should be some way of doing this with custom __getstate__ and __setstate__ methods. It's just tricky to get right and a bit fragile. It's much easier to write the migration code if both the old and new class are separate and functioning at the same time. The main problem is that the advertised API says you should do: from plone.registry import Record from plone.registry import field registry['foo.bar'] = Record(field.TextLine(), umy value) Here, field.TextLine derives from PersistentField which derives from Persistent, and Record derives from Persistent also. If I wanted to get rid of the Persistent base, I'd have to make a new tree of field types (the standard zope.schema ones still need some subclassing), and a new Record class with a less obvious name. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Changing and migrating persistence structure
Hi, [I posted this to zodb-dev, but it seems that list isn't working at the moment(?) so I thought I'd try here too] I have a package (plone.registry) that currently has a persistent structure like this: Registry(Persistent) | +-- Records(Persistent) | +-- BTree of Record(Persistent) | +-- PersistentField(Persistent) That is, a Registry is a persistent object containing a persistent Records object that in turn contains a BTree of persistent Record objects that contain a persistent PersistentField and a primitive value. This is quite inefficient, though, because it results in a lot of object loads. On any given request, some of our projects load a dozen or more values from the registry. Each is just a simple primitive, but we need to load the full shebang to make it work. Now, I'd like to move to this structure: Registry(Persistent) | +-- Records | +-- BTree of Field | +-- BTree of values Here, there's only one Persistent object, plus the two BTrees: one holding all the fields and one holding all the values. Records no longer needs to be persistent (its attributes become part of the parent Registry's _p_jar). Fields no longer need to be persistent either, since they are in effect immutable objects. Values are primitives anyway. I've done this (in a branch) and it works for new sites. However, I'm having a nightmare trying to migrate old sites. As soon as I access anything that uses the registry, I get ZODB errors, because the persistent structure is now different. In particular, it's trying to read a value into e.g. a Records object that used to derive from Persistent, but now no longer does. What is the best way to manage this type of migration? In terms of API compatibility, I'd really like to keep plone.registry.Record as the name and module path of the record, since it is used by the API. The difference is that before it was persisted and returned by an API on the Registry. Now, it's constructed as needed on the fly from the internal data structure. The same applies to the various field types that derive from PersistentField, which are now Persistent, but won't be. There's code and documentation out there that use these. I'm less worried about the Records object, which was always an implementation detail, and the BTree-of-records, which will never have been accessed directly. Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] docs.zope.org automation
On 2 August 2010 22:40, Jens Vagelpohl j...@dataflake.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 8/2/10 16:36 , Stephan Richter wrote: On Monday, August 02, 2010, Jens Vagelpohl wrote: 'll have to look at that. Currently, the documentation builder does not do any introspection on the package itself, mostly because I do not want to fully install the package and pull in all dependencies. Maybe there's a simple way that does not require full installation. I agree. This does not build the package: python setup.py --long-description Thanks for the hint, I'll try that. Can you give me a sample package where the long description is supposed to be the main documentation? And what's the output from that? If it's ReST I'd have to find a way to convert it to HTML on the fly... sigh z3c.form, I'd guess. :) For other examples, look at e.g. z3c.caching, plone.caching or plone.testing. You can do: $ python setup.py --long-description | rst2html.py doc.html to create an HTML file. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 19:26, yuppie y.2...@wcm-solutions.de wrote: Hi! Traditionally the last two lines of unit test files look like this: if __name__ == '__main__': unittest.main(defaultTest='test_suite') That makes it easy to run the tests of a specific file. But it doesn't work with tests that require the zope testrunner. AFAICS something like this is needed instead: if __name__ == '__main__': from zope.testing.testrunner import run run(['-m', 'test_foo', '--test-path', '.']) Questions: -- 1.) Is it still policy to add these lines? 2.) Is there a better solution for using zope testrunner than the one shown above? I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 20:14, yuppie y.2...@wcm-solutions.de wrote: Martin Aspeli wrote: I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically. Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? See the -s and -t options. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 22:35, Tres Seaver tsea...@palladion.com wrote: I don't believe that zope.testing's testrunner works without 'def test_suite()'. Latter versions can detect unittest.TestCase-derived test suites automatically. For doctests you still need test_suite(). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Should we merge collective.xmltestreport into zope.testrunner?
Hi, On 25 July 2010 16:58, Hanno Schlichting ha...@hannosch.eu wrote: On Sun, Jul 25, 2010 at 5:36 AM, Martin Aspeli optilude+li...@gmail.com wrote: A while back, I wrote collective.xmltestreport [1,2]. In short, it's a wrapper around zope.testing's test runner that can produce output in an XML format compatible with the xUnit family of testing tools. This is useful for integrating with things like Hudson, which can parse these kinds of files. The problem is that changes in zope.testing/zope.testrunner keep breaking collective.xmltestreport. At first, I was using -x as the option to turn on XML output, before zope.testing took that option as its own. Now, with version 3.10, it's broken again, because it's trying to import zope.testing.testrunner, which is no longer there. I wonder if it'd be a good idea to just merge this functionality into zope.testrunner itself? We recently merged in support for subunit [0], which claims to have amongst others subunit2junitxml - convert a subunit stream to a JUnit XML representation.. Cool! That sounds to me very much like the same functionality. Having two ways of getting the same result seems suboptimal to me. But I haven't looked into subunit yet, so I'm not sure if it is cross-platform and easily installable. I'd love to see someone write up some documentation on how to use it inside Hudson ;) Definitely. collective.xmltestreport is a bit clunky anyway. If we have something better, I'm all for it. [0] https://launchpad.net/subunit/ Are there any docs on how this actually works? I see a changelog entry for zope.testing, but no information on how to actually use it. :( Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Should we merge collective.xmltestreport into zope.testrunner?
Hi, A while back, I wrote collective.xmltestreport [1,2]. In short, it's a wrapper around zope.testing's test runner that can produce output in an XML format compatible with the xUnit family of testing tools. This is useful for integrating with things like Hudson, which can parse these kinds of files. The problem is that changes in zope.testing/zope.testrunner keep breaking collective.xmltestreport. At first, I was using -x as the option to turn on XML output, before zope.testing took that option as its own. Now, with version 3.10, it's broken again, because it's trying to import zope.testing.testrunner, which is no longer there. I wonder if it'd be a good idea to just merge this functionality into zope.testrunner itself? The problem is that in the short term, I probably won't have time to look into it. I may, but there are a few other things that have to take precedence. I was going to just leave it, but given that people are working on zope.testing/zope.testrunner right now, I thought I'd bring it up and see if there was any interest. I'd be happy to support anyone who wants to attempt this, and in reality it's only a couple of relatively small modules that need to be transposed into the codebase. It'd need better (read: some) test coverage if it were to go into zope.testrunner, though. Is there interest in this kind of thing? Martin [1] http://pypi.python.org/pypi/collective.xmltestreport [2] http://svn.plone.org/svn/collective/collective.xmltestreport/trunk ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-Checkins] SVN: Zope/branches/2.12/doc/CHANGES.rst Correct the past.
Log message for revision 114489: Correct the past. Changed: U Zope/branches/2.12/doc/CHANGES.rst -=- Modified: Zope/branches/2.12/doc/CHANGES.rst === --- Zope/branches/2.12/doc/CHANGES.rst 2010-07-10 10:00:41 UTC (rev 114488) +++ Zope/branches/2.12/doc/CHANGES.rst 2010-07-10 10:02:50 UTC (rev 114489) @@ -15,9 +15,8 @@ - LP #143531: Fix broken object so they give access to their state. -- LP #578326: Issue a warning if someone specifies a non-public permission - attribute in the browser:view directive. This attribute has never been - supported in Zope 2. +- LP #578326: Add support for non-public permission attributes in the + browser:view directive. Features Added ++ ___ Zope-Checkins maillist - Zope-Checkins@zope.org https://mail.zope.org/mailman/listinfo/zope-checkins
[Zope-Checkins] SVN: Zope/trunk/src/Products/Five/browser/ Merge r114488 from 2.12 branch
Log message for revision 114490:
Merge r114488 from 2.12 branch
Changed:
U Zope/trunk/src/Products/Five/browser/metaconfigure.py
U Zope/trunk/src/Products/Five/browser/tests/pages.txt
U Zope/trunk/src/Products/Five/browser/tests/pages.zcml
-=-
Modified: Zope/trunk/src/Products/Five/browser/metaconfigure.py
===
--- Zope/trunk/src/Products/Five/browser/metaconfigure.py 2010-07-10
10:02:50 UTC (rev 114489)
+++ Zope/trunk/src/Products/Five/browser/metaconfigure.py 2010-07-10
10:13:30 UTC (rev 114490)
@@ -201,15 +201,7 @@
):
if permission is None:
permission = 'zope.Public'
-elif permission in ('zope.Public', 'zope2.Public'):
-# No need to warn about the default case
-pass
-else:
-warnings.warn(The permission option of the browser:view /
- directive is not supported in Zope 2. + \
- Ignored for %s in %s %
- (str(class_), _context.info), stacklevel=3)
-
+
super(view, self).__init__(
_context, permission, for_=for_, name=name, layer=layer,
class_=class_, allowed_interface=allowed_interface,
@@ -314,6 +306,42 @@
newclass, (for_, layer), self.provides, name,
_context.info),
)
+
+# Security
+
+_context.action(
+discriminator = ('five:protectClass', newclass),
+callable = protectClass,
+args = (newclass, permission)
+)
+
+if allowed_attributes:
+for attr in allowed_attributes:
+_context.action(
+discriminator = ('five:protectName', newclass, attr),
+callable = protectName,
+args = (newclass, attr, permission)
+)
+
+# Make everything else private
+allowed = allowed_attributes or []
+private_attrs = [name for name in dir(newclass)
+ if (not name.startswith('_')) and
+(name not in allowed) and
+ismethod(getattr(newclass, name))]
+for attr in private_attrs:
+_context.action(
+discriminator = ('five:protectName', newclass, attr),
+callable = protectName,
+args = (newclass, attr, CheckerPrivateId)
+)
+
+# Protect the class
+_context.action(
+discriminator = ('five:initialize:class', newclass),
+callable = InitializeClass,
+args = (newclass,)
+)
_factory_map = {'image':{'prefix':'ImageResource',
'count':0,
Modified: Zope/trunk/src/Products/Five/browser/tests/pages.txt
===
--- Zope/trunk/src/Products/Five/browser/tests/pages.txt2010-07-10
10:02:50 UTC (rev 114489)
+++ Zope/trunk/src/Products/Five/browser/tests/pages.txt2010-07-10
10:13:30 UTC (rev 114490)
@@ -253,12 +253,34 @@
aq_parent(aq_inner(context))
Folder at /test_folder_1_
+The same applies to a view registered with browser:view / instead of
+browser:page /
+
+ request = TestRequest()
+ view = getMultiAdapter((self.folder.testoid, request),
name=u'permission_view')
+ view.__ac_permissions__
+ (('View management screens', ('',)),)
+ aq_acquire(view, '__roles__')
+ ('Manager',)
+ context = view.context
+ from Acquisition import ImplicitAcquisitionWrapper
+ type(context) == ImplicitAcquisitionWrapper
+ True
+ view.__parent__ == view.context
+ True
+ aq_parent(view) == view.context
+ True
+ context.aq_inner.aq_parent
+ Folder at /test_folder_1_
+ aq_parent(aq_inner(context))
+ Folder at /test_folder_1_
+
High-level security
---
protected_view_names = [
... 'eagle.txt', 'falcon.html', 'owl.html', 'flamingo.html',
- ... 'condor.html']
+ ... 'condor.html', 'permission_view']
public_view_names = [
... 'public_attribute_page',
Modified: Zope/trunk/src/Products/Five/browser/tests/pages.zcml
===
--- Zope/trunk/src/Products/Five/browser/tests/pages.zcml 2010-07-10
10:02:50 UTC (rev 114489)
+++ Zope/trunk/src/Products/Five/browser/tests/pages.zcml 2010-07-10
10:13:30 UTC (rev 114490)
@@ -232,7 +232,15 @@
class=.pages.SimpleView
permission=zope2.Public
/
-
+
+ !-- A named view with permissions --
+ browser:view
+ name=permission_view
+ for=Products.Five.tests.testing.simplecontent.ISimpleContent
+ class=.pages.CallView
+ permission=zope2.ViewManagementScreens
+ /
+
!-- stuff that we'll override in overrides.zcml --
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 27 June 2010 00:24, Hanno Schlichting ha...@hannosch.eu wrote: Hi there, recently MJ opened a security related bug and disclosed it to the public at https://bugs.launchpad.net/zope2/+bug/578326. In short Zope 2 never supported the permission attribute on ZCML browser:view declarations. It seems some people might have specified this attribute and assumed it would do something. I have added a warning message to Zope 2 (trunk + 2.12 branch) which warns about those cases. This is similar to how we handle other such cases like the unsupported require set_schema=.. / and require set_attributes=... / on class directives. But it turns out that Zope 2 itself is using this in one place, that looks like it ought to have a security declaration. The Products.Five.adding.ContentAdding class registered as an add view (+) has no working security declarations I can see, and only has such a non-functioning permission=zope2.ViewManagementScreens set. I'm not familiar enough with the add view concept to understand what this is doing. It also looks like both CMF and Plone use similar registrations for their add views. Ideally I'd love to add support for the permission attribute, as clearly people have been using it. But if there's nobody who can figure out how to do that, I'd at least like to clarify the add view case. Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm allowed to close the issue on Launchpad, but it should be fine now. Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 10 July 2010 18:16, Hanno Schlichting ha...@hannosch.eu wrote: On Sat, Jul 10, 2010 at 12:14 PM, Martin Aspeli optilude+li...@gmail.com wrote: Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm allowed to close the issue on Launchpad, but it should be fine now. Awesome! You truly rock! My powers of copy and paste and sharp, it must be said. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 9 July 2010 16:12, Hanno Schlichting ha...@hannosch.eu wrote: On Thu, Jul 8, 2010 at 3:02 PM, Martin Aspeli optilude+li...@gmail.com wrote: Ideally I'd love to add support for the permission attribute, as clearly people have been using it. But if there's nobody who can figure out how to do that, I'd at least like to clarify the add view case. Why can't we just copy the relevant code from the browser:page directive? The ViewSecurityGrokker in http://svn.zope.org/five.grok/trunk/src/five/grok/meta.py?rev=112163view=auto may be useful reading too. It should be doing the same thing, no? It seems you have some idea about this code, so are you volunteering to implement this? Possibly. I have client work that has to take priority right now. Since we are dealing with a disclosed real security vulnerability here, I need to have some resolution by next Tuesday. Either that is disabling the functionality or protecting it with some security. I'd appreciate it if someone who's getting more than four hours of sleep a night at the moment takes a stab. I'm happy to review/assist. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
Hi Hanno, On 27 June 2010 00:24, Hanno Schlichting ha...@hannosch.eu wrote: Hi there, recently MJ opened a security related bug and disclosed it to the public at https://bugs.launchpad.net/zope2/+bug/578326. In short Zope 2 never supported the permission attribute on ZCML browser:view declarations. It seems some people might have specified this attribute and assumed it would do something. I have added a warning message to Zope 2 (trunk + 2.12 branch) which warns about those cases. This is similar to how we handle other such cases like the unsupported require set_schema=.. / and require set_attributes=... / on class directives. But it turns out that Zope 2 itself is using this in one place, that looks like it ought to have a security declaration. The Products.Five.adding.ContentAdding class registered as an add view (+) has no working security declarations I can see, and only has such a non-functioning permission=zope2.ViewManagementScreens set. I'm not familiar enough with the add view concept to understand what this is doing. It also looks like both CMF and Plone use similar registrations for their add views. And Dexterity, I suggest. Ideally I'd love to add support for the permission attribute, as clearly people have been using it. But if there's nobody who can figure out how to do that, I'd at least like to clarify the add view case. Why can't we just copy the relevant code from the browser:page directive? The ViewSecurityGrokker in http://svn.zope.org/five.grok/trunk/src/five/grok/meta.py?rev=112163view=auto may be useful reading too. It should be doing the same thing, no? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New release of z3c.form
On 1 July 2010 21:10, Godefroid Chapelle got...@bubblenet.be wrote: Le 01/07/10 14:53, Stephan Richter a écrit : On Thursday, July 01, 2010, Godefroid Chapelle wrote: I found http://docs.zope.org/z3c.form/ Last updated Nov 08,2009 Version 1.8.2dev. What is the process to get the latest version published ? Not sure. I think Paul Carduner did that. Since z3c.form is Sphinx-enabled, I wonder whether we could stick the docs into http://packages.python.org/z3c.form. Manuel does this already: http://packages.python.org/manuel/ Regards, Stephan Done http://packages.python.org/z3c.form Who will get rid of http://docs.zope.org/z3c.form/ ? If you do, can you please make it link or redirect to the new location? There are live links to that kind of thing in lots of documentation I've written. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-Checkins] SVN: Zope/branches/2.12/ Fix processInputs() so that it no longer stomps on things like :records or :int:list
Log message for revision 112780:
Fix processInputs() so that it no longer stomps on things like :records or
:int:list
Changed:
U Zope/branches/2.12/doc/CHANGES.rst
U Zope/branches/2.12/src/Products/Five/browser/decode.py
U Zope/branches/2.12/src/Products/Five/browser/tests/test_decode.py
-=-
Modified: Zope/branches/2.12/doc/CHANGES.rst
===
--- Zope/branches/2.12/doc/CHANGES.rst 2010-05-27 12:12:28 UTC (rev 112779)
+++ Zope/branches/2.12/doc/CHANGES.rst 2010-05-27 13:27:15 UTC (rev 112780)
@@ -11,6 +11,10 @@
Bugs Fixed
++
+- Five's processInputs() would stomp on :list or :tuple values that contained
+ ints or other non-strings, would clear out :records entirely, and would not
+ do anything for :record fields.
+
- LP #143261: The (very old-fashioned) Zope2.debug interactive request
debugger still referred to the toplevel module ``Zope``, which was
renamed to ``Zope2`` a long time ago.
Modified: Zope/branches/2.12/src/Products/Five/browser/decode.py
===
--- Zope/branches/2.12/src/Products/Five/browser/decode.py 2010-05-27
12:12:28 UTC (rev 112779)
+++ Zope/branches/2.12/src/Products/Five/browser/decode.py 2010-05-27
13:27:15 UTC (rev 112780)
@@ -32,23 +32,40 @@
pass
return text
+def processInputValue(value, charsets):
+Recursively look for values (e.g. elements of lists, tuples or dicts)
+and attempt to decode.
+
+
+if isinstance(value, list):
+return [processInputValue(v, charsets) for v in value]
+elif isinstance(value, tuple):
+return tuple([processInputValue(v, charsets) for v in value])
+elif isinstance(value, dict):
+for k, v in value.items():
+value[k] = processInputValue(v, charsets)
+return value
+elif isinstance(value, str):
+return _decode(value, charsets)
+else:
+return value
+
def processInputs(request, charsets=None):
+Process the values in request.form to decode strings to unicode, using
+the passed-in list of charsets. If none are passed in, look up the user's
+preferred charsets. The default is to use utf-8.
+
+
if charsets is None:
-envadapter = IUserPreferredCharsets(request)
-charsets = envadapter.getPreferredCharsets() or ['utf-8']
-
+envadapter = IUserPreferredCharsets(request, None)
+if envadapter is None:
+charsets = ['utf-8']
+else:
+charsets = envadapter.getPreferredCharsets() or ['utf-8']
+
for name, value in request.form.items():
if not (isCGI_NAME(name) or name.startswith('HTTP_')):
-if isinstance(value, str):
-request.form[name] = _decode(value, charsets)
-elif isinstance(value, list):
-request.form[name] = [ _decode(val, charsets)
- for val in value
- if isinstance(val, str) ]
-elif isinstance(value, tuple):
-request.form[name] = tuple([ _decode(val, charsets)
- for val in value
- if isinstance(val, str) ])
+request.form[name] = processInputValue(value, charsets)
def setPageEncoding(request):
Set the encoding of the form page via the Content-Type header.
Modified: Zope/branches/2.12/src/Products/Five/browser/tests/test_decode.py
===
--- Zope/branches/2.12/src/Products/Five/browser/tests/test_decode.py
2010-05-27 12:12:28 UTC (rev 112779)
+++ Zope/branches/2.12/src/Products/Five/browser/tests/test_decode.py
2010-05-27 13:27:15 UTC (rev 112780)
@@ -46,6 +46,42 @@
processInputs(request, charsets)
request.form['foo'] == (u'f\xf6\xf6',)
True
+
+Ints in lists are not lost::
+
+ request.form['foo'] = [1, 2, 3]
+ processInputs(request, charsets)
+ request.form['foo'] == [1, 2, 3]
+ True
+
+Ints in tuples are not lost::
+
+ request.form['foo'] = (1, 2, 3,)
+ processInputs(request, charsets)
+ request.form['foo'] == (1, 2, 3)
+ True
+
+Mixed lists work:
+
+ request.form['foo'] = [u'f\xf6\xf6'.encode('iso-8859-1'), 2, 3]
+ processInputs(request, charsets)
+ request.form['foo'] == [u'f\xf6\xf6', 2, 3]
+ True
+
+Mixed dicts work:
+
+ request.form['foo'] = {'foo': u'f\xf6\xf6'.encode('iso-8859-1'),
'bar': 2}
+ processInputs(request, charsets)
+ request.form['foo'] == {'foo': u'f\xf6\xf6', 'bar': 2}
+ True
+
+Deep recursion works:
+
+ request.form['foo'] = [{'foo': u'f\xf6\xf6'.encode('iso-8859-1'),
'bar': 2}, {'foo': uone, 'bar': 3}]
+ processInputs(request, charsets)
+
[Zope-Checkins] SVN: Zope/trunk/src/Products/Five/browser/ Merge c112780 from 2.12 branch
Log message for revision 112781:
Merge c112780 from 2.12 branch
Changed:
U Zope/trunk/src/Products/Five/browser/decode.py
U Zope/trunk/src/Products/Five/browser/tests/test_decode.py
-=-
Modified: Zope/trunk/src/Products/Five/browser/decode.py
===
--- Zope/trunk/src/Products/Five/browser/decode.py 2010-05-27 13:27:15 UTC
(rev 112780)
+++ Zope/trunk/src/Products/Five/browser/decode.py 2010-05-27 13:30:02 UTC
(rev 112781)
@@ -32,23 +32,40 @@
pass
return text
+def processInputValue(value, charsets):
+Recursively look for values (e.g. elements of lists, tuples or dicts)
+and attempt to decode.
+
+
+if isinstance(value, list):
+return [processInputValue(v, charsets) for v in value]
+elif isinstance(value, tuple):
+return tuple([processInputValue(v, charsets) for v in value])
+elif isinstance(value, dict):
+for k, v in value.items():
+value[k] = processInputValue(v, charsets)
+return value
+elif isinstance(value, str):
+return _decode(value, charsets)
+else:
+return value
+
def processInputs(request, charsets=None):
+Process the values in request.form to decode strings to unicode, using
+the passed-in list of charsets. If none are passed in, look up the user's
+preferred charsets. The default is to use utf-8.
+
+
if charsets is None:
-envadapter = IUserPreferredCharsets(request)
-charsets = envadapter.getPreferredCharsets() or ['utf-8']
-
+envadapter = IUserPreferredCharsets(request, None)
+if envadapter is None:
+charsets = ['utf-8']
+else:
+charsets = envadapter.getPreferredCharsets() or ['utf-8']
+
for name, value in request.form.items():
if not (isCGI_NAME(name) or name.startswith('HTTP_')):
-if isinstance(value, str):
-request.form[name] = _decode(value, charsets)
-elif isinstance(value, list):
-request.form[name] = [ _decode(val, charsets)
- for val in value
- if isinstance(val, str) ]
-elif isinstance(value, tuple):
-request.form[name] = tuple([ _decode(val, charsets)
- for val in value
- if isinstance(val, str) ])
+request.form[name] = processInputValue(value, charsets)
def setPageEncoding(request):
Set the encoding of the form page via the Content-Type header.
Modified: Zope/trunk/src/Products/Five/browser/tests/test_decode.py
===
--- Zope/trunk/src/Products/Five/browser/tests/test_decode.py 2010-05-27
13:27:15 UTC (rev 112780)
+++ Zope/trunk/src/Products/Five/browser/tests/test_decode.py 2010-05-27
13:30:02 UTC (rev 112781)
@@ -46,6 +46,42 @@
processInputs(request, charsets)
request.form['foo'] == (u'f\xf6\xf6',)
True
+
+Ints in lists are not lost::
+
+ request.form['foo'] = [1, 2, 3]
+ processInputs(request, charsets)
+ request.form['foo'] == [1, 2, 3]
+ True
+
+Ints in tuples are not lost::
+
+ request.form['foo'] = (1, 2, 3,)
+ processInputs(request, charsets)
+ request.form['foo'] == (1, 2, 3)
+ True
+
+Mixed lists work:
+
+ request.form['foo'] = [u'f\xf6\xf6'.encode('iso-8859-1'), 2, 3]
+ processInputs(request, charsets)
+ request.form['foo'] == [u'f\xf6\xf6', 2, 3]
+ True
+
+Mixed dicts work:
+
+ request.form['foo'] = {'foo': u'f\xf6\xf6'.encode('iso-8859-1'),
'bar': 2}
+ processInputs(request, charsets)
+ request.form['foo'] == {'foo': u'f\xf6\xf6', 'bar': 2}
+ True
+
+Deep recursion works:
+
+ request.form['foo'] = [{'foo': u'f\xf6\xf6'.encode('iso-8859-1'),
'bar': 2}, {'foo': uone, 'bar': 3}]
+ processInputs(request, charsets)
+ request.form['foo'] == [{'foo': u'f\xf6\xf6', 'bar': 2}, {'foo':
uone, 'bar': 3}]
+ True
+
def test_suite():
___
Zope-Checkins maillist - Zope-Checkins@zope.org
https://mail.zope.org/mailman/listinfo/zope-checkins
Re: [Zope-dev] z3c.form release
On 15 May 2010 15:39, Wichert Akkerman wich...@wiggy.net wrote: On 5/10/10 17:24 , Wichert Akkerman wrote: I fixed a few issues in z3c.form today. Can anyone make a 2.3.4 release? Since I got no reaction I'll repeat this request: can someone please make a new z3c.form release? Looking at the pypi page there is no shortage of people with the ability to do so.. And now you're one of those people. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Hanno, please update the ZTK
On 4 May 2010 00:09, Martijn Faassen faas...@startifact.com wrote: Hanno is making releases of packages in the ZTK. So it's not just Hanno's waste of time; it's mine too. That's where I was coming from when this discussion started. It didn't help that the action of making the fork really hurt me at the time - I'm not inclined to be calm about it. Unfortunately, that probably means you're going to be ignored. I'm not saying that to spite you. It's a dispassionate evaluation of what's going on right now. If I could, I'd try to get you, Hanno, Lennart, Chris McDonough and a large amount of beer in the same room. I don't think this is going to get anywhere by email. I don't think anyone even knows what this really is. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.test.doctest made into monkey-patches (Was: Circular dependency hell.)
On 20 April 2010 21:23, Lennart Regebro rege...@gmail.com wrote: On Tue, Apr 20, 2010 at 13:44, Wichert Akkerman wich...@wiggy.net wrote: You may want to move it outside the zope.* namespace to encourage that :) -1 I think zope.testrunner is just fine, and acknowledges the heritage. Namespaces should be about which community (or company) owns a package, not marketing. I think we're a little over-sensitive to the it's Zope so we hate it sentiment. The people (if any) who still have such childish ideas are probably not all that interesting to us as consumers of our software anyway. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Circular dependency hell.
Hi Christian, On 21 April 2010 02:58, Christian Theune c...@gocept.com wrote: On 04/20/2010 08:44 PM, Jim Fulton wrote: On Tue, Apr 20, 2010 at 12:09 PM, Christian Theunec...@gocept.com wrote: Minor note: zope.testing *promotes* layers the wrong way and zope.app.testing definitely implements them the wrong way. That's prety vague. Could you say specifically in what ways zope.testing promotes layers the wrong way and zope.testing uses the attribute '__bases__' to store the information what the base layers are. __*__ are supposedly Python internal attributes. Specifically __bases__ is known to be used to store information which base classes a class has. Looking at this I (and others too) get directed towards: aha, so layers are classes and use inheritance to signal what base layers are. Which is exactly not what is happening. In fact, it's a little worse than that. Consider this pair of layers: class Base: @classmethod def setUp(cls): print Setting up base @classmethod def tearDown(cls): print Tearing down base class Child(Base): @classmethod def setUp(cls): print Setting up child Note that there's no tearDown on the child (perhaps it doesn't need one). What actually happens in this case is that the test runner still finds a tearDown on Child, it's just that it's inherited from Base. So in effect, Base's tearDown is called twice. This also happens with things like testSetUp() and testTearDown(). If the base defines them and a child doesn't, they're called twice. The other problem is that it's hard to also use inheritance in the OOP sense to re-use layer logic. Also, if the layer manages any state, it has to be set as a class variable (on cls), which is effectively global. If you want to re-use a layer but isolate the resources its creates from those created by existing layers, you have to re-implement the layer. These insights by Ross Patterson led to collective.testcaselayer, which was lightly refactored into the layer module of the nascent plone.testing. See: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.py http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.txt http://svn.plone.org/svn/plone/plone.testing/trunk/README.txt This module also contains an implementation of a resource manager that allows layers to define shared resources in a stack that lets child layers shadow those resources (i.e. provide a changed fixture). We use this for things like ZODB connections and Zope 2 app roots. It's explained best in the README, and tested in layer.txt. Having used this pattern for a while, I'm pretty sure it's an improvement on the layers-are-classes thing, which in addition to the problems above, has caused a fair amount of confusion. zope.app.testing uses them the wrong way? Actually it doesn't. I confused this with Zope 2's Testing: There's Testing/ZopeTestCase/layer.py which defines a class with classmethods and in a similar fashion there is Products.PloneTestCase that defines classes, derives them and thus kind of piggybacks on the class inheritance mechanism to establish __bases__ paired with static methods but without actually inheriting methods. FTR, the ZopeTestCase mess is basically what plone.testing.z2 tries to fix (insofar as it's possible). The PloneTestCase mess will hopefully be fixed by a plone.app.testing building on plone.testing. We struggled through some hairy details that I fail to remember when we worked on gocept.selenium last year which tries to establish a generic layer that can be combined with others. You're not the only one. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 18 April 2010 05:20, Tres Seaver tsea...@palladion.com wrote: I'm not against having the snippets be executable, because I *do* want them to work. I just don't want to encourage anyone to think that they are testing the software when they write the snippets, or execute them. Executing the snippets is testing the documentation, not the software. Everyone who's ever written a doctest, or is thinking about doing so: Please frame this and hang it above your monitor. :-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Circular dependency hell.
Hi Lennart co, On 17 April 2010 02:38, Lennart Regebro rege...@gmail.com wrote: On Fri, Apr 16, 2010 at 19:53, Jonathan Lange j...@mumak.net wrote: As the author of one of those other testrunners, I can tell you that if you do this you'll find that your number one biggest problem is getting layers to work. Ah, right, layers are in zope.testing too. To actually get widespread movement from zope.testing we would have to make some other layer support. Hm... As you may know, I've been working on plone.testing. This is mainly to make it easier for people working on Plone packages to write good tests (and a lot of it is just about patterns, rather than code), but in fact it doesn't depend on Plone (and only soft-depends on Zope 2). I'm certainly hoping to use it for my plain Zope 3/Toolkit packages in the future. plone.testing makes very heavy use of layers. I think layers are a great feature, when done properly, and I haven't seen any better approach. The setUpClass/setUpModule stuff in unittest2 is nice, but doesn't really solve the same problem. For integration testing with something as complex as Zope 2 (or, arguably, the ZODB, various bits of the ZTK, and so on) layers allow us the framework authors to make life much easier for those people who are not experts in the framework. Anyway, a few high level observations: - In neither plone.testing (apart from its own tests), nor in code that uses it, would I imagine actually depending on zope.testing via an import. We use unittest(2) and doctest from the standard library. - We do depend on a zope.testing-compatible test runner, in that we expect layers to work. In reality, we depend on zc.recipe.testrunner, though I would *love* to be able to do 'python setup.py test' as well (and have that execute tests with layers). I have no idea how that works or whether it'd be possible. - The way zope.testing promotes writing layers is actually pretty evil. It overloads the 'class' keyword, essentially, making you spell base layers as base classes. This has a few problems: - If your base layer has a testTearDown method, say, and your layer doesn't, the base class version will actually inherit into the child layer. zope.testing will then run the same code twice, once for the base layer and once for the child layer. - You can't use OOP inheritance to re-use layer conveniences. - People get quite confused. :) - You can't have two copies of a layer - all layers are module-level global singletons. This becomes somewhat important when layers manage and expose resources (like database connections). Anyway, based on Ross Patterson's work in collective.testcaselayer, I think we have something better in plone.testing's layer module. General info and patterns: http://svn.plone.org/svn/plone/plone.testing/trunk/README.txt Layer module: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.py Layer doctests: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.txt If I could have my cake and eat it too, I'd like: - A test runner that supports layers - A test runner that supports the new fixture methods in unittest2/Python 2.7 - setUpClass and setUpModule - A test runner that properly reports skipped tests (pretty easy) - A test runner that supports the above begin run from 'setup.py test' - A test runner installable via a buildout part just like zc.recipe.testrunner We could quite possibly factor layer.py out of plone.testing and push it into something else if that was desirable. It's self-contained and has no dependencies. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 17 April 2010 09:41, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This kind of goes with Lennart's frustration about trying to port the ZTK packages, or a core subset, to Python 3. I would like to see the ZTK packages have really excellent documentation, as well as 100% test coverage. Thoughts? +1000, to both aim and approach. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-Checkins] SVN: Zope/branches/2.12/src/Products/Five/browser/resource.py Apply some safety to the fix in r110185 after reports it may be causing some problems
Log message for revision 110799: Apply some safety to the fix in r110185 after reports it may be causing some problems Changed: U Zope/branches/2.12/src/Products/Five/browser/resource.py -=- Modified: Zope/branches/2.12/src/Products/Five/browser/resource.py === --- Zope/branches/2.12/src/Products/Five/browser/resource.py2010-04-13 14:27:23 UTC (rev 110798) +++ Zope/branches/2.12/src/Products/Five/browser/resource.py2010-04-13 14:43:07 UTC (rev 110799) @@ -27,6 +27,7 @@ from zope.app.publisher.fileresource import File, Image from zope.app.publisher.pagetemplateresource import PageTemplate +from Acquisition import aq_base from Products.Five.browser import BrowserView @@ -164,7 +165,8 @@ # We need to propagate security so that restrictedTraverse() will # work -resource.__roles__ = self.__roles__ +if hasattr(aq_base(self), '__roles__'): +resource.__roles__ = self.__roles__ return resource ___ Zope-Checkins maillist - Zope-Checkins@zope.org https://mail.zope.org/mailman/listinfo/zope-checkins
[Zope-Checkins] SVN: Zope/trunk/src/Products/Five/browser/resource.py Merge c110799 from svn+ssh://svn.zope.org/repos/main/Zope/branches/2.12
Log message for revision 110800: Merge c110799 from svn+ssh://svn.zope.org/repos/main/Zope/branches/2.12 Changed: U Zope/trunk/src/Products/Five/browser/resource.py -=- Modified: Zope/trunk/src/Products/Five/browser/resource.py === --- Zope/trunk/src/Products/Five/browser/resource.py2010-04-13 14:43:07 UTC (rev 110799) +++ Zope/trunk/src/Products/Five/browser/resource.py2010-04-13 14:47:33 UTC (rev 110800) @@ -27,6 +27,7 @@ from zope.publisher.interfaces.browser import IBrowserPublisher from zope.ptresource.ptresource import PageTemplate +from Acquisition import aq_base from Products.Five.browser import BrowserView @@ -165,7 +166,8 @@ # We need to propagate security so that restrictedTraverse() will # work -resource.__roles__ = self.__roles__ +if hasattr(aq_base(self), '__roles__'): +resource.__roles__ = self.__roles__ return resource ___ Zope-Checkins maillist - Zope-Checkins@zope.org https://mail.zope.org/mailman/listinfo/zope-checkins
[Zope-Checkins] SVN: Zope/trunk/src/ZServer/PubCore/__init__.py A hint for the next person who gets very confused when importing something a second time doesn't work.
Log message for revision 110660: A hint for the next person who gets very confused when importing something a second time doesn't work. Changed: U Zope/trunk/src/ZServer/PubCore/__init__.py -=- Modified: Zope/trunk/src/ZServer/PubCore/__init__.py === --- Zope/trunk/src/ZServer/PubCore/__init__.py 2010-04-08 15:54:42 UTC (rev 110659) +++ Zope/trunk/src/ZServer/PubCore/__init__.py 2010-04-08 15:55:42 UTC (rev 110660) @@ -24,6 +24,8 @@ return apply(_handle, args, kw) def setNumberOfThreads(n): +This function will self-destruct in 4 statements. + global _n _n=n global setNumberOfThreads ___ Zope-Checkins maillist - Zope-Checkins@zope.org https://mail.zope.org/mailman/listinfo/zope-checkins
[Zope-dev] Stacking zope.component registries
Hi, I'd like to come up with a way to set up a test fixture that does the component registry equivalent of stackable DemoStorage's: whilst a layer is in effect, calls to provideAdapter() and friends (and ZCML execution) go into a global registry that is stacked on top of the default global one. On layer tear-down, the registry is popped, leaving the original (or previous) global registry intact. In zope.component.globalregistry, I see: def provideUtility(component, provides=None, name=u''): base.registerUtility(component, provides, name, event=False) base is a module-level variable of type BaseGlobalComponents(). I guess there'd be a way to use __bases__ to create this kind of stack, but I'm not clear on the details, and in particular whether this would really work with provideAdapter() and the rest of the (test-oriented) Python API. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Stacking zope.component registries
Stephan Richter wrote: On Thursday 08 April 2010, Marius Gedminas wrote: Someone (I'm bad with names, sorry!) recently proposed a change to zope.configuration that makes ZCML directives use getSiteManager() instead of getGlobalSiteManager(); with that patch in, Chris's example should make ZCML configuration register all the components into your stacked registry. Although you'd have the other problem of setSite() having no effect on the site manager, which would break all local utilities and adapters in your tests. What about using z3c.baseregistry? You can wrap the baseregistry call around your ZCML. This way you can create on registry per layer and then use the __bases__ attribute to stack them together. Yes, I did look at it. However, the real goal is to provide isolation for anything that makes ZCA registrations. In particular, that includes provideAdapter() and friends. I suspect z3c.baseregistry can't deal with this? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Stacking zope.component registries
Chris McDonough wrote: On 4/8/10 4:36 PM, Marius Gedminas wrote: from zope.component import getSiteManager getSiteManager.sethook(get_current_registry) That seems a bit short-sighted: it would break all tests that rely on setSite() working. He said he wanted a global registry, but.. who knows? I stay as far away from that API as possible. ;-) Marius is right - I need standard stuff like setSite(), local components and that pesky global API (provideAdapter etc.) to work. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Stacking zope.component registries
Stephan Richter wrote: On Thursday 08 April 2010, Martin Aspeli wrote: Yes, I did look at it. However, the real goal is to provide isolation for anything that makes ZCA registrations. In particular, that includes provideAdapter() and friends. I suspect z3c.baseregistry can't deal with this? I think it can be done, just not in the way I intended it to. Instead of adding the base registry to bases, you have to make it the actual registry for the API, which should be no problem from a functional test setup function. Can you elaborate on what you mean here? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Stacking zope.component registries
Stephan Richter wrote:
On Thursday 08 April 2010, Martin Aspeli wrote:
Can you elaborate on what you mean here?
So I think this can all be done independently of base registry (unless you are
are planning to store the registry in the ZODB).
The key for layering is the ability to inherit components using the __bases__
attribute (see registry.py). So something like:
reg1 = Components('reg1') # Registry for layer 1
Then for the next layer that is based on layer 1, you can do:
reg2 = Components('reg2', (reg1,)) # Registry for layer 2
This should behave identically to how ZODB storage layering works.
Right. However, any calls to provideAdapter() and friends would still
use the global registry, unless I monkey patch
zope.component.globalregistry.base, as would any ZCML directives, I guess.
Martin
--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Fred Drake wrote: On Tue, Apr 6, 2010 at 10:25 PM, Martin Aspelioptilude+li...@gmail.com wrote: So this is still in pyexpat C code as far as I can tell. :-( This is saddening. But on the other hand, your dedication in helping me find a fix is heartening. ;) I've not managed a 64-bit sandbox, which I suspect is what I really need to debug that. Will shoot for this weekend, since last didn't pan out. I have a slight suspicion that lxml is involved somewhere. I've managed to make it crash reliably simply by doing a parse of an XML file, and I think such a parse may be happening as a side effect of a module import caused by that very ZCML file. That doesn't really explain why pdb dies when it dies, but I don't know how the C/Python boundary is working here (especially since I can't really see the C code). I'm re-building for the n-th time, so I'll check again tomorrow when that's done. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Martin Aspeli wrote:
Hi,
I'm not sure if this is a Python issue or a zope issue. We're getting a
segfault on 64-bit SuSE Linux (SLES 11), originating from
z3c.autoinclude, which in turn called zope.configuration'sinclude /
implementation. This calls expat, which then crashes (no error, log
message, or core file, but it has all the markings of a segfault) during
parsing of the file.
The weird thing is that it's not parsing of any file: it happens during
a standard configure.zcml (for the collective.xdv package), but
z3c.autoinclude itself is invoked from another ZCML file, so it must've
been able to read that.
I tried running it through gdb, and got:
(gdb) run
Starting program: /home/osc/python-env/bin/python2.6 ./bin/instance1 fg
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
2010-04-07 09:38:22 INFO ZServer HTTP server started at Wed Apr 7
09:38:22 2010
Hostname: 0.0.0.0
Port: 8801
/home/osc/osc/eggs/collective.wtf-1.0b9-py2.6.egg/collective/wtf/exportimport.py:8:
DeprecationWarning: InitializeClass is deprecated. import from
App.class_init instead
from Globals import InitializeClass
Program exited normally.
.. which is not very helpful.
The status code is indeed zero.
(python-env)o...@lwpn-osb-webback-2:~/osc ./bin/instance1 fg2010-04-07
09:39:21 INFO ZServer HTTP server started at Wed Apr 7 09:39:21 2010
Hostname: 0.0.0.0
Port: 8801
/home/osc/osc/eggs/collective.wtf-1.0b9-py2.6.egg/collective/wtf/exportimport.py:8:
DeprecationWarning: InitializeClass is deprecated. import from
App.class_init instead
from Globals import InitializeClass
(python-env)o...@lwpn-osb-webback-2:~/osc echo $?
0
Next, I tried to pdb from roughly where I know it dies. Sorry about the
long text here. It's a bit weird, but I'll explain what's happening.
(python-env)o...@lwpn-osb-webback-2:~/osc ./bin/instance1 fg2010-04-07
09:47:37 INFO ZServer HTTP server started at Wed Apr 7 09:47:37 2010
Hostname: 0.0.0.0
Port: 8801
This is where the brekapoint was set, in z3c.autoinclude's
includePlugins / handler. This looks for packages with a particular
entry point and loads their meta.zcml, then configure.zcml. The problem
happens during configure.zcml loading.
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(104)includePluginsDirective()
- for filename in zcml_to_look_for:
snip /
We've now stepped into the function that does the real work:
- def includeZCMLGroup(_context, info, filename, override=False):
(Pdb) l
10from z3c.autoinclude.plugin import PluginFinder
11
12import logging
13log = logging.getLogger(z3c.autoinclude)
14
15 -def includeZCMLGroup(_context, info, filename, override=False):
16includable_zcml = info[filename]
17# ^^ a key error would mean that we are trying to include a
group of ZCML
18#with a filename that wasn't ever searched for. that
*should* be an error
19
20zcml_context = repr(_context.info)
snip /
This quickly gets to here, where it's loading a package and calling
zope.configuration's include / handler.
(Pdb) l
22for dotted_name in includable_zcml:
23log.debug('including file %s from package %s at %s',
filename, dotted_name, zcml_context)
24
25for dotted_name in includable_zcml:
26includable_package = resolve(dotted_name)
27 -if override:
28includeOverrides(_context, filename, includable_package)
29else:
30include(_context, filename, includable_package)
31
32
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(30)includeZCMLGroup()
- include(_context, filename, includable_package)
It's not na override, so we end up in zope.configuration now.
--Call--
/home/osc/osc/eggs/zope.configuration-3.6.0-py2.6.egg/zope/configuration/xmlconfig.py(487)include()
- def include(_context, file=None, package=None, files=None):
(Pdb) n
/home/osc/osc/eggs/zope.configuration-3.6.0-py2.6.egg/zope/configuration/xmlconfig.py(493)include()
- if files:
(Pdb) l
488 Include a zcml file
489
490 See examples in tests/text_xmlconfig.py
491
492
493 - if files:
494 if file:
495 raise ValueError(Must specify only one of file or files)
496 elif not file:
497 file = 'configure.zcml'
498
(Pdb) l
499 # BBB 2006/12/19 -- to be removed after 12 months
500 # This is a backward-compatibility support for old site.conf
501
502 if package and (package.__name__ == 'zope.app'):
503
Re: [Zope-dev] Segfault in zope.configuration
Martin Aspeli wrote:
At this point, something is printed to the console. collective.wtf is a
dependency of lw.portal, and its ZCML is being included from lw.portal.
/home/osc/osc/eggs/collective.wtf-1.0b9-py2.6.egg/collective/wtf/exportimport.py:8:
DeprecationWarning: InitializeClass is deprecated. import from
App.class_init instead
from Globals import InitializeClass
Okay, I found out what it does next: It now processes z3c.autoinclude's
includeDependencies /, which basically just goes through the
dependencies in setup.py's install_requires and tries to load a
configure.zcml for each. It successfully does that for a number of
dependencies, up until it hits collective.xdv (same package I saw the
problem with when I set up a minimal buildout that just had Plone and
collective.xdv in it).
The pdb log is:
(Pdb) l
43
44def includeDependenciesDirective(_context, package):
45
46import pdb; pdb.set_trace()
47
48 -if api.dependencies_disabled():
49log.warn('z3c.autoinclude.dependency is disabled but is
being invoked by %s' % _context.info)
50return
51
52dist = distributionForPackage(package)
53info =
DependencyFinder(dist).includableInfo(['configure.zcml', 'meta.zcml'])
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(52)includeDependenciesDirective()
- dist = distributionForPackage(package)
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(53)includeDependenciesDirective()
- info = DependencyFinder(dist).includableInfo(['configure.zcml',
'meta.zcml'])
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(55)includeDependenciesDirective()
- includeZCMLGroup(_context, info, 'meta.zcml')
(Pdb) pp info
{'configure.zcml': ['collective.xdv', 'five.grok', 'z3c.jbot'],
'meta.zcml': ['five.grok', 'z3c.jbot']}
(Pdb) l
50return
51
52dist = distributionForPackage(package)
53info =
DependencyFinder(dist).includableInfo(['configure.zcml', 'meta.zcml'])
54
55 -includeZCMLGroup(_context, info, 'meta.zcml')
56includeZCMLGroup(_context, info, 'configure.zcml')
57
58def includeDependenciesOverridesDirective(_context, package):
59
60if api.dependencies_disabled():
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(56)includeDependenciesDirective()
- includeZCMLGroup(_context, info, 'configure.zcml')
(Pdb) s
--Call--
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(15)includeZCMLGroup()
- def includeZCMLGroup(_context, info, filename, override=False):
(Pdb) n
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(16)includeZCMLGroup()
- includable_zcml = info[filename]
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(20)includeZCMLGroup()
- zcml_context = repr(_context.info)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(23)includeZCMLGroup()
- log.debug('including file %s from package %s at %s', filename,
dotted_name, zcml_context)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(23)includeZCMLGroup()
- log.debug('including file %s from package %s at %s', filename,
dotted_name, zcml_context)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(23)includeZCMLGroup()
- log.debug('including file %s from package %s at %s', filename,
dotted_name, zcml_context)
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(22)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(25)includeZCMLGroup()
- for dotted_name in includable_zcml:
(Pdb)
/home/osc/osc/eggs/z3c.autoinclude-0.3.2-py2.6.egg/z3c/autoinclude/zcml.py(26)includeZCMLGroup()
- includable_package = resolve(dotted_name)
(Pdb) l
21
22for dotted_name in includable_zcml:
23log.debug('including file %s from package %s at %s',
filename, dotted_name, zcml_context)
24
25for dotted_name in includable_zcml:
26 -includable_package = resolve(dotted_name)
27if override:
28includeOverrides(_context, filename, includable_package)
29else:
30include(_context, filename
[Zope-dev] zope.testing and unittest2
Hi, Has anyone given any thought to supporting the new constructs of unittest2 (http://pypi.python.org/pypi/unittest2) in zope.testing? Using zope.testing 3.9.3 and a simple test case with unittest2, I made the following observations: - A basic test case works fine; a subclass of unittest2.TestCase is picked up for tests - The new assertions work. - The @skip, @skipIf, @skipUnless and @expectedFailure decorators work. However, there is a deprecation warning: Use of a TestResult without an addExpectedFailure method is deprecated. In the output, zope.testing reports Ran 2 tests with 0 failures and 0 errors when running one skipped and one passing test. - setUpClass()/tearDownClass() methods are not executed - Neither are setUpModule()/tearDownModule() methods Overall, it seems that we'd need to: - Make zope.testing aware of skipped tests and report these separately. - Call the class and module level fixture methods. I'm not entirely sure how these would interact with layers. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.13 roadmap
Hanno Schlichting wrote: On Thu, Apr 1, 2010 at 3:49 AM, Martin Aspelioptilude+li...@gmail.com wrote: What's the next step? I'd love to see some roadmapping ala that you did for Plone 5, in particular to discuss our WSGI story (which I'm interested in helping out with if others can help too). There's no big roadmap yet, but I have some ideas :) In general I'd like to avoid a major Zope release, that isn't used by an official Plone release. Zope 2.11 hasn't seen much attention and we had to maintain 2.10 anyways. But if there's a decent and stable feature set that other consumers like to get there hands on, we can get them a release. I just won't be pushing into that direction. Here's what I can see today: There's a bunch of stuff already done as noted in http://svn.zope.org/Zope/trunk/doc/CHANGES.rst?view=markup. - zope.app removal This project is all finished. Zope2 doesn't contain any trace of zope.app or the name Zope 3 in itself nor its dependencies anymore. Cool. - Moving formlib out of the core Triggered by the above and finished as well. Zope trunk doesn't contain any trace of formlib anymore. The relevant code is now in the five.formlib package. This package is also included in Zope 2.12, so you can start using the functionality and be compatible with both Zope versions. CMF does this already. Cool. - WSGI I cannot tell at the moment. It's going to depend on the actual changes required to get this to work. Since there is some broken WSGI support inside the 2.12 codebase, we can claim a certain deal of changes as bugfixes. But there's limits to what we can warrant as a bugfix. If the code changes are self-contained and only touch the broken WSGI modules, we are good. If we require changes all over the board for whatever reasons, this will have to be a new feature and go into Zope 2.13. Only a branch with actual code will tell :) It'd be interesting to see what comes out of the PSU sprint Tres talked about. I have an interest in this, so will be happy to help, though I'll not be able to drive it entirely. - ZTK As long as there's no formal release of the ZTK or a defined and stable process around it, Zope 2 defines its own KGS. It so happens to use exactly the same versions as the current ZTK trunk. Once we are making progress on the ZTK release, we'll see how Zope 2 can use such a release. My current guess is that Zope 2.13 will use some ZTK 1.1 release. +1 - I think there's a big win from this. I also hope we can squash all the what is the ZTK debates. At the end, it feels like a lot of bike-shedding. - Five deprecation I did a whole bunch of work on this already and keep working on it. The end goal is to be able to deprecate the entire Products.Five and phase it out of the core. Actually useful functionality in it, is integrated into the proper places inside the Zope2 core packages instead. Like security stuff in AccessControl, container events in OFS, reading site.zcml and setting up the CA in Zope2/App and so forth. There's a number of hard cases, where there's some semantic differences between zope.* packages and their Five equivalent, especially in the browser realm. This project might not be completely finished for 2.13. And yes, there's some difficult questions with non-obvious answers around this. We'll deal with them once we get there. I'm focussing on the obvious parts first. +1 - sounds hairy, though. Getting rid of the Zope 2 specific ViewPageTemplateFile and BrowserView (already done, I guess) would be a good starting point. - Reduce C code in Zope 2 My first part of this was discussed and implemented. The remaining C code inside the Zope 2 distribution is in AccessControl, DocumentTemplate and ZCTextIndex. We'll see what to do about them, once their packages are actually self-contained. I'd also like a more sane/documented/debuggable AccessControl. Sigh. - Make Zope 2 more modular This is related to the above two points. I'm not quite sure about the details of the implementation yet. But in general it would be nice, if a consumer of Zope 2 could use it's core, without getting a whole bunch of stuff it doesn't want. The obvious example here is Plone, which continues to use less and less code from Zope 2, but has no chance of making a radical cut and loose it all. There's interesting problems, like being able to use Zope 2 without the ZMI. In some sense this is similar to using the ZTK instead of zope.app. The only thing I know for sure, is that I'd like to first make the packages inside Zope 2 standalone and reusable and only once they are, break them into more distributions. We've gone the other way around with Zope 3 and it has cost a whole lot of pain. I think it may help to make a list of the things we'd like to be able to get rid of, and then isolate those. - ZODB 3.10 Jim promised a second alpha release to be out shortly. Should be low risk to upgrade to it. The
[Zope-dev] Segfault in zope.configuration
Hi, I'm not sure if this is a Python issue or a zope issue. We're getting a segfault on 64-bit SuSE Linux (SLES 11), originating from z3c.autoinclude, which in turn called zope.configuration's include / implementation. This calls expat, which then crashes (no error, log message, or core file, but it has all the markings of a segfault) during parsing of the file. The weird thing is that it's not parsing of any file: it happens during a standard configure.zcml (for the collective.xdv package), but z3c.autoinclude itself is invoked from another ZCML file, so it must've been able to read that. Any tips on how to debug or similar experiences would be appreciated! Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Segfault in zope.configuration
Fred Drake wrote: On Thu, Apr 1, 2010 at 6:07 AM, Martin Aspelioptilude+li...@gmail.com wrote: Any tips on how to debug or similar experiences would be appreciated! If you're on some Unix flavor, you should be able to deconstruct the return code from the runzope process to determine if the app was killed by a signal and, if so, which. If it's a segfault, you should be looking for signal.SIGSEGV. I can't check until after the long weekend now, but I'm pretty sure it is. I'd be very interested in hearing if Expat is implicated. I'm pretty sure it is. The pdb rabbit hole ended at pyexpat.c. I can't see what's going on there, but when I did 'r' it blew up. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
