Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 17 April 2010 09:41, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > This kind of goes with Lennart's frustration about trying to port the > ZTK packages, or a core subset, to Python 3. > > I would like to see the ZTK packages have really excellent > documentation, as well as 100% test coverage. > Thoughts? +1000, to both aim and approach. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 18 April 2010 05:20, Tres Seaver wrote: > I'm not against having the snippets be executable, because I *do* want > them to work. I just don't want to encourage anyone to think that they > are testing the software when they write the snippets, or execute them. > Executing the snippets is testing the documentation, not the software. Everyone who's ever written a doctest, or is thinking about doing so: Please frame this and hang it above your monitor. :-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.test.doctest made into monkey-patches (Was: Circular dependency hell.)
On 20 April 2010 21:23, Lennart Regebro wrote: > On Tue, Apr 20, 2010 at 13:44, Wichert Akkerman wrote: >> You may want to move it outside the zope.* namespace to encourage that :) -1 I think zope.testrunner is just fine, and acknowledges the heritage. Namespaces should be about which community (or company) owns a package, not marketing. I think we're a little over-sensitive to the "it's Zope so we hate it" sentiment. The people (if any) who still have such childish ideas are probably not all that interesting to us as consumers of our software anyway. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Circular dependency hell.
Hi Christian, On 21 April 2010 02:58, Christian Theune wrote: > On 04/20/2010 08:44 PM, Jim Fulton wrote: >> >> On Tue, Apr 20, 2010 at 12:09 PM, Christian Theune wrote: >>> >>> Minor note: zope.testing *promotes* layers the wrong way and >>> zope.app.testing definitely implements them the wrong way. >> >> That's prety vague. Could you say specifically in what ways >> zope.testing promotes layers the wrong way and > > zope.testing uses the attribute '__bases__' to store the information what > the base layers are. __*__ are supposedly Python internal attributes. > Specifically __bases__ is known to be used to store information which base > classes a class has. > > Looking at this I (and others too) get directed towards: aha, so layers are > classes and use inheritance to signal what base layers are. Which is exactly > not what is happening. In fact, it's a little worse than that. Consider this pair of layers: class Base: @classmethod def setUp(cls): print "Setting up base" @classmethod def tearDown(cls): print "Tearing down base" class Child(Base): @classmethod def setUp(cls): print "Setting up child" Note that there's no tearDown on the child (perhaps it doesn't need one). What actually happens in this case is that the test runner still finds a tearDown on Child, it's just that it's inherited from Base. So in effect, Base's tearDown is called twice. This also happens with things like testSetUp() and testTearDown(). If the base defines them and a child doesn't, they're called twice. The other problem is that it's hard to also use inheritance in the OOP sense to re-use layer logic. Also, if the layer manages any state, it has to be set as a class variable (on cls), which is effectively global. If you want to re-use a layer but isolate the resources its creates from those created by existing layers, you have to re-implement the layer. These insights by Ross Patterson led to collective.testcaselayer, which was lightly refactored into the layer module of the nascent plone.testing. See: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.py http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.txt http://svn.plone.org/svn/plone/plone.testing/trunk/README.txt This module also contains an implementation of a resource manager that allows layers to define shared resources in a stack that lets child layers shadow those resources (i.e. provide a changed fixture). We use this for things like ZODB connections and Zope 2 app roots. It's explained best in the README, and tested in layer.txt. Having used this pattern for a while, I'm pretty sure it's an improvement on the layers-are-classes thing, which in addition to the problems above, has caused a fair amount of confusion. > > zope.app.testing uses them the wrong way? > > Actually it doesn't. I confused this with Zope 2's Testing: > > There's Testing/ZopeTestCase/layer.py which defines a class with > classmethods and in a similar fashion there is Products.PloneTestCase that > defines classes, derives them and thus kind of piggybacks on the class > inheritance mechanism to establish __bases__ paired with static methods but > without actually inheriting methods. FTR, the ZopeTestCase mess is basically what plone.testing.z2 tries to fix (insofar as it's possible). The PloneTestCase mess will hopefully be fixed by a plone.app.testing building on plone.testing. > We struggled through some hairy details that I fail to remember when we > worked on gocept.selenium last year which tries to establish a generic layer > that can be combined with others. You're not the only one. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Hanno, please update the ZTK
On 4 May 2010 00:09, Martijn Faassen wrote: > Hanno is making releases of packages in the ZTK. So it's not just > Hanno's waste of time; it's mine too. That's where I was coming from > when this discussion started. It didn't help that the action of making > the fork really hurt me at the time - I'm not inclined to be calm > about it. Unfortunately, that probably means you're going to be ignored. I'm not saying that to spite you. It's a dispassionate evaluation of what's going on right now. If I could, I'd try to get you, Hanno, Lennart, Chris McDonough and a large amount of beer in the same room. I don't think this is going to get anywhere by email. I don't think anyone even knows what "this" really is. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] z3c.form release
On 15 May 2010 15:39, Wichert Akkerman wrote: > On 5/10/10 17:24 , Wichert Akkerman wrote: > > I fixed a few issues in z3c.form today. Can anyone make a 2.3.4 release? > > > Since I got no reaction I'll repeat this request: can someone please > make a new z3c.form release? Looking at the pypi page there is no > shortage of people with the ability to do so.. > And now you're one of those people. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New release of z3c.form
On 1 July 2010 21:10, Godefroid Chapelle wrote: > Le 01/07/10 14:53, Stephan Richter a écrit : >> On Thursday, July 01, 2010, Godefroid Chapelle wrote: >>> I found http://docs.zope.org/z3c.form/ >>> >>> Last updated Nov 08,2009 Version 1.8.2dev. >>> >>> What is the process to get the latest version published ? >> >> Not sure. I think Paul Carduner did that. Since z3c.form is Sphinx-enabled, I >> wonder whether we could stick the docs into >> http://packages.python.org/z3c.form. >> >> Manuel does this already: >> >> http://packages.python.org/manuel/ >> >> Regards, >> Stephan > > Done > > http://packages.python.org/z3c.form > > Who will get rid of http://docs.zope.org/z3c.form/ ? If you do, can you please make it link or redirect to the new location? There are "live" links to that kind of thing in lots of documentation I've written. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
Hi Hanno,
On 27 June 2010 00:24, Hanno Schlichting wrote:
> Hi there,
>
> recently MJ opened a security related bug and disclosed it to the
> public at https://bugs.launchpad.net/zope2/+bug/578326.
>
> In short Zope 2 never supported the permission attribute on ZCML
> browser:view declarations. It seems some people might have specified
> this attribute and assumed it would do something.
>
> I have added a warning message to Zope 2 (trunk + 2.12 branch) which
> warns about those cases. This is similar to how we handle other such
> cases like the unsupported and set_attributes="..." /> on class directives.
>
> But it turns out that Zope 2 itself is using this in one place, that
> looks like it ought to have a security declaration. The
> Products.Five.adding.ContentAdding class registered as an add view
> ("+") has no working security declarations I can see, and only has
> such a non-functioning permission="zope2.ViewManagementScreens" set.
> I'm not familiar enough with the add view concept to understand what
> this is doing. It also looks like both CMF and Plone use similar
> registrations for their add views.
And Dexterity, I suggest.
> Ideally I'd love to add support for the permission attribute, as
> clearly people have been using it. But if there's nobody who can
> figure out how to do that, I'd at least like to clarify the add view
> case.
Why can't we just copy the relevant code from the browser:page directive?
The ViewSecurityGrokker in
http://svn.zope.org/five.grok/trunk/src/five/grok/meta.py?rev=112163&view=auto
may be useful reading too. It should be doing the same thing, no?
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 9 July 2010 16:12, Hanno Schlichting wrote: > On Thu, Jul 8, 2010 at 3:02 PM, Martin Aspeli > wrote: >>> Ideally I'd love to add support for the permission attribute, as >>> clearly people have been using it. But if there's nobody who can >>> figure out how to do that, I'd at least like to clarify the add view >>> case. >> >> Why can't we just copy the relevant code from the browser:page directive? >> >> The ViewSecurityGrokker in >> http://svn.zope.org/five.grok/trunk/src/five/grok/meta.py?rev=112163&view=auto >> may be useful reading too. It should be doing the same thing, no? > > It seems you have some idea about this code, so are you volunteering > to implement this? Possibly. I have client work that has to take priority right now. > Since we are dealing with a disclosed real security vulnerability > here, I need to have some resolution by next Tuesday. Either that is > disabling the functionality or protecting it with some security. I'd appreciate it if someone who's getting more than four hours of sleep a night at the moment takes a stab. I'm happy to review/assist. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 27 June 2010 00:24, Hanno Schlichting wrote:
> Hi there,
>
> recently MJ opened a security related bug and disclosed it to the
> public at https://bugs.launchpad.net/zope2/+bug/578326.
>
> In short Zope 2 never supported the permission attribute on ZCML
> browser:view declarations. It seems some people might have specified
> this attribute and assumed it would do something.
>
> I have added a warning message to Zope 2 (trunk + 2.12 branch) which
> warns about those cases. This is similar to how we handle other such
> cases like the unsupported and set_attributes="..." /> on class directives.
>
> But it turns out that Zope 2 itself is using this in one place, that
> looks like it ought to have a security declaration. The
> Products.Five.adding.ContentAdding class registered as an add view
> ("+") has no working security declarations I can see, and only has
> such a non-functioning permission="zope2.ViewManagementScreens" set.
> I'm not familiar enough with the add view concept to understand what
> this is doing. It also looks like both CMF and Plone use similar
> registrations for their add views.
>
> Ideally I'd love to add support for the permission attribute, as
> clearly people have been using it. But if there's nobody who can
> figure out how to do that, I'd at least like to clarify the add view
> case.
Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm
allowed to close the issue on Launchpad, but it should be fine now.
Cheers,
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 10 July 2010 18:16, Hanno Schlichting wrote: > On Sat, Jul 10, 2010 at 12:14 PM, Martin Aspeli > wrote: >> Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm >> allowed to close the issue on Launchpad, but it should be fine now. > > Awesome! You truly rock! My powers of copy and paste and sharp, it must be said. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] z3c.form: Problem with DictionaryField datamanger
On 11 July 2010 01:56, Michael Howitz wrote: > Hi Wichert, > > In Revision r112225 (see [1]) you make sure that the DictionaryField data > manager conforms to the interface by raising an exception in > datamanager.get() when a requested key is not found in the underlying > dictionary. > > Although it is nice to conform to the interface, this changes breaks code in > a project of mine: z3c.form.form.applyChanges uses datamanager.get() to check > whether the value has changed. Before r112225 this call returned the > field.missing_value when the requested key does not exist. Now it raises an > AttributeError, so applyChanges cannot continue. > > I'm not sure whether it was intended that now all keys must exist in the > dictionary. I think this should be consistent with AttributeField. This does: return getattr(self.adapted_context, self.field.__name__) which would indeed raise an AttributeError if the field doesn't exist. > In r114527 I prepared a change on a branch which uses datamanager.query() get > the old value for comparison which allows that keys can be missing again in > the dictionary. This seems like the use case query() was designed for... > I'd like to merge this change (it breaks no tests) into the trunk. Any > objections? +1 for this change Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Should we merge collective.xmltestreport into zope.testrunner?
Hi, A while back, I wrote collective.xmltestreport [1,2]. In short, it's a wrapper around zope.testing's test runner that can produce output in an XML format compatible with the xUnit family of testing tools. This is useful for integrating with things like Hudson, which can parse these kinds of files. The problem is that changes in zope.testing/zope.testrunner keep breaking collective.xmltestreport. At first, I was using "-x" as the option to turn on XML output, before zope.testing took that option as its own. Now, with version 3.10, it's broken again, because it's trying to import zope.testing.testrunner, which is no longer there. I wonder if it'd be a good idea to just merge this functionality into zope.testrunner itself? The problem is that in the short term, I probably won't have time to look into it. I may, but there are a few other things that have to take precedence. I was going to just leave it, but given that people are working on zope.testing/zope.testrunner right now, I thought I'd bring it up and see if there was any interest. I'd be happy to support anyone who wants to attempt this, and in reality it's only a couple of relatively small modules that need to be transposed into the codebase. It'd need better (read: some) test coverage if it were to go into zope.testrunner, though. Is there interest in this kind of thing? Martin [1] http://pypi.python.org/pypi/collective.xmltestreport [2] http://svn.plone.org/svn/collective/collective.xmltestreport/trunk ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Should we merge collective.xmltestreport into zope.testrunner?
Hi, On 25 July 2010 16:58, Hanno Schlichting wrote: > On Sun, Jul 25, 2010 at 5:36 AM, Martin Aspeli > wrote: >> A while back, I wrote collective.xmltestreport [1,2]. In short, it's a >> wrapper around zope.testing's test runner that can produce output in >> an XML format compatible with the xUnit family of testing tools. This >> is useful for integrating with things like Hudson, which can parse >> these kinds of files. >> >> The problem is that changes in zope.testing/zope.testrunner keep >> breaking collective.xmltestreport. At first, I was using "-x" as the >> option to turn on XML output, before zope.testing took that option as >> its own. Now, with version 3.10, it's broken again, because it's >> trying to import zope.testing.testrunner, which is no longer there. >> >> I wonder if it'd be a good idea to just merge this functionality into >> zope.testrunner itself? > > We recently merged in support for subunit [0], which claims to have > amongst others "subunit2junitxml - convert a subunit stream to a JUnit > XML representation.". Cool! > That sounds to me very much like the same functionality. Having two > ways of getting the same result seems suboptimal to me. But I haven't > looked into subunit yet, so I'm not sure if it is cross-platform and > easily installable. I'd love to see someone write up some > documentation on how to use it inside Hudson ;) Definitely. collective.xmltestreport is a bit clunky anyway. If we have something better, I'm all for it. > [0] https://launchpad.net/subunit/ Are there any docs on how this actually works? I see a changelog entry for zope.testing, but no information on how to actually use it. :( Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 19:26, yuppie wrote: > Hi! > > > Traditionally the last two lines of unit test files look like this: > > if __name__ == '__main__': > unittest.main(defaultTest='test_suite') > > That makes it easy to run the tests of a specific file. But it doesn't > work with tests that require the zope testrunner. AFAICS something like > this is needed instead: > > if __name__ == '__main__': > from zope.testing.testrunner import run > run(['-m', 'test_foo', '--test-path', '.']) > > > Questions: > -- > > 1.) Is it still policy to add these lines? > > 2.) Is there a better solution for using zope testrunner than the one > shown above? I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 20:14, yuppie wrote: > Martin Aspeli wrote: >> I never do either. I install zc.recipe.testrunner in a buildout and >> use bin/test, which picks up tests in modules automatically. > > Sure. But do you always run all tests it picks up while working on a > specific test file? Or do you use bin/test with options that allow to > run specific files? See the -s and -t options. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 22:35, Tres Seaver wrote: > I don't believe that zope.testing's testrunner works without 'def > test_suite()'. Latter versions can detect unittest.TestCase-derived test suites automatically. For doctests you still need test_suite(). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] docs.zope.org automation
On 2 August 2010 22:40, Jens Vagelpohl wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 8/2/10 16:36 , Stephan Richter wrote: >> On Monday, August 02, 2010, Jens Vagelpohl wrote: >>> 'll have to look at that. Currently, the documentation builder does not >>> do any introspection on the package itself, mostly because I do not want >>> to fully install the package and pull in all dependencies. Maybe there's >>> a simple way that does not require full installation. >> >> I agree. This does not build the package: >> >> python setup.py --long-description > > Thanks for the hint, I'll try that. Can you give me a sample package > where the long description is supposed to be the main documentation? And > what's the output from that? If it's ReST I'd have to find a way to > convert it to HTML on the fly... z3c.form, I'd guess. :) For other examples, look at e.g. z3c.caching, plone.caching or plone.testing. You can do: $ python setup.py --long-description | rst2html.py > doc.html to create an HTML file. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] z3c.blobfile release and PyPI ownership
Hi, I'd like to make a new z3c.blobfile release. Can I please have PyPI access? The current owners are uoestermeier, nadako. Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Changing and migrating persistence structure
Hi, [I posted this to zodb-dev, but it seems that list isn't working at the moment(?) so I thought I'd try here too] I have a package (plone.registry) that currently has a persistent structure like this: Registry(Persistent) | +--> Records(Persistent) | +--> BTree of Record(Persistent) | +--> PersistentField(Persistent) That is, a Registry is a persistent object containing a persistent Records object that in turn contains a BTree of persistent Record objects that contain a persistent PersistentField and a primitive value. This is quite inefficient, though, because it results in a lot of object loads. On any given request, some of our projects load a dozen or more values from the registry. Each is just a simple primitive, but we need to load the full shebang to make it work. Now, I'd like to move to this structure: Registry(Persistent) | +--> Records | +--> BTree of Field | +--> BTree of values Here, there's only one Persistent object, plus the two BTrees: one holding all the fields and one holding all the values. Records no longer needs to be persistent (its attributes become part of the parent Registry's _p_jar). Fields no longer need to be persistent either, since they are in effect immutable objects. Values are primitives anyway. I've done this (in a branch) and it works for new sites. However, I'm having a nightmare trying to migrate old sites. As soon as I access anything that uses the registry, I get ZODB errors, because the persistent structure is now different. In particular, it's trying to read a value into e.g. a Records object that used to derive from Persistent, but now no longer does. What is the best way to manage this type of migration? In terms of API compatibility, I'd really like to keep plone.registry.Record as the name and module path of the record, since it is used by the API. The difference is that before it was persisted and returned by an API on the Registry. Now, it's constructed as needed on the fly from the internal data structure. The same applies to the various field types that derive from PersistentField, which are now Persistent, but won't be. There's code and documentation out there that use these. I'm less worried about the Records object, which was always an implementation detail, and the BTree-of-records, which will never have been accessed directly. Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Changing and migrating persistence structure
Hi Jim,
On 08/08/2010, Jim Fulton wrote:
> On Thu, Aug 5, 2010 at 2:36 AM, Martin Aspeli
> wrote:
> ...
>> I have a package (plone.registry) that currently has a persistent
>> structure like this:
>>
>> Registry(Persistent)
>> |
>> +--> Records(Persistent)
>> |
>> +--> BTree of Record(Persistent)
>> |
>> +--> PersistentField(Persistent)
>>
>> That is, a Registry is a persistent object containing a persistent
>> Records object that in turn contains a BTree of persistent Record
>
> Since BTrees are mapping, I assume that you mean the values are
> records and that the keys are something boring like strings or
> integers.
Yes. The keys are strings.
> I like to use mathematical notation when talking about BTrees and
> sets, as in:
>
> Registry
> BTree {? -> Record}
>
>
>> objects that contain a persistent PersistentField and a primitive
>> value.
>>
>> This is quite inefficient, though, because it results in a lot of
>> object loads. On any given request, some of our projects load a dozen
>> or more values from the registry. Each is just a simple primitive, but
>> we need to load the full shebang to make it work.
>
> Not sure what you mean by "full shebang".
The Registry, Records object, the relevant Record in the relevant
BTree, and possibly the PersistentField object.
In the "new" branch it just looks up the value in the relevant BTree.
>> Now, I'd like to move to this structure:
>>
>> Registry(Persistent)
>> |
>> +--> Records
>> |
>> +--> BTree of Field
>> |
>> +--> BTree of values
>
> I'm foggy on what "field" and "value" are here or what your queries
> are doing. Maybe this is just a distraction.
Somewhat, unless you've worked with plone.registry. The point is to
allow the "get a value" API to just look at self.values[key], which is
a fast lookup and doesn't load anything except the relevant BTree
bucket + the registry itself.
>> Here, there's only one Persistent object, plus the two BTrees: one
>> holding all the fields and one holding all the values. Records no
>> longer needs to be persistent (its attributes become part of the
>> parent Registry's _p_jar).
>
> I wonder what role "Records" plays independent of the "Registry".
None, really. The main reason to have it is to be able to have an API
like registry.records with dict-like notation (there's also
__getitem__ on the registry, which returns the value of a given key,
not the Record). I made ``records`` an attribute of type Records, and
Records derives from Persistent. I wish I hadn't, since it can just
live in its parent's _p_jar.
> I also wonder why it matters whether it is persistent or not.
It's better if it isn't (one fewer object to load/fill up the cache),
though the real culprits are the many Record objects each being
persistent and loaded separately.
On a given request, we can end up loading a dozen or more values from
the registry, which means a dozen or more objects in the cache and
associated overhead.
>> Fields no longer need to be persistent
>> either, since they are in effect immutable objects. Values are
>> primitives anyway.
>>
>> I've done this (in a branch) and it works for new sites. However, I'm
>> having a nightmare trying to migrate old sites. As soon as I access
>> anything that uses the registry, I get ZODB errors, because the
>> persistent structure is now different. In particular, it's trying to
>> read a value into e.g. a Records object that used to derive from
>> Persistent, but now no longer does.
>
> What savings do you get by making Records non-persistent?
One fewer persistent object.
I think the real saving is in making the Record object non-persistent,
especially since the "read" use case can just read from the ``values``
BTree with the structure above.
>> What is the best way to manage this type of migration?
>
> Today, it probably makes the most sense to make new classes for the
> non-persistemnt objects. You'll then need to write a script to
> rebuild the data structures.
Okay. So there's no way to get at the data if I take Persistent out of
the base classes for Records / Record.
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Changing and migrating persistence structure
On 8 August 2010 20:29, Hanno Schlichting wrote: > There should be some way of doing this with custom __getstate__ and > __setstate__ methods. > > It's just tricky to get right and a bit fragile. It's much easier to > write the migration code if both the old and new class are separate > and functioning at the same time. The main problem is that the advertised API says you should do: from plone.registry import Record from plone.registry import field registry['foo.bar'] = Record(field.TextLine(), u"my value") Here, field.TextLine derives from PersistentField which derives from Persistent, and Record derives from Persistent also. If I wanted to get rid of the Persistent base, I'd have to make a new "tree" of field types (the standard zope.schema ones still need some subclassing), and a new Record class with a less obvious name. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: ETag support for zope.browserresource
On 10 August 2010 02:25, Marius Gedminas wrote: > I've added ETag support for zope.browserresource in a branch: > http://zope3.pov.lt/trac/changeset/115596 > > Does anybody have any comments/objections? If not, I'd like to merge > this to trunk and release zope.browserresource 3.11.0. No strong objections, really, but bear in mind that people will likely want to customise this. With plone.caching / plone.app.caching we have a framework that, among other things, deals with browser resources and sets etag, last-modified and other headers according to rules configured by the developer and/or administrator. I think that would just stomp on etags set by zope.browserresource, but worth bearing in mind that for bigger applications like Plone, we need a centralised, overarching and configurable strategy for cache headers. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] (Zope2) a proposed tweak to assigning default roles to permissions
Hi, On 19 August 2010 16:46, Hanno Schlichting wrote: > Hi. > > On Thu, Aug 19, 2010 at 6:15 AM, David Glick > wrote: >> As an alternative to requiring calling setDefaultRoles/addPermission at >> import time, I suggest that we add an optional roles attribute to the >> directive. This would then be used when the directive is >> executed, instead of the current hard-coded Manager setting. Examples: >> >> >> > roles="Manager SiteAdmin"/> >> >> > > Can roles currently contain whitespace? Like "Awesome People"? Yes, they can. > If so, we should go for nested nodes: > > > Manager > SiteAdmin > Awesome People > > > I think this matches the style of some of the GenericSetup handlers > which deal with permissions. +1 Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] .lock files on Windows
Hi, With Plone 4 and thus Zope 2.12.10, we've noticed a problem that I think only affects Windows. Can anyone confirm or shed some more light? Basically, if we run an instance (installed via plone.recipe.zope2instance as bin\instance) in the foreground (bin\instance fg) and then kill it with Ctrl+C, the var\instance.lock and var\instance.pid files are not cleaned up. These need to be manually deleted, otherwise Zope refuses to start up with the confusing message "Please stop the instance before starting it". Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] .lock files on Windows
On 3 September 2010 17:06, Jim Pharis wrote: > I'm on 2.13.0a3 w/plone.recipe.zope2instanec-4.0.2. Under the scenario > Martin described, exiting a fg with ctrl-c, the lock file is cleaned up for > me. If I kill the service using Task Manager the lock file remains. It seemed to be happening intermittently. But it happened on two different PCs running Windows XP. Instances were started with "bin\instance fg" and killed with Ctrl+C. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 3 Newbie Feedback
Hi, So how to make Blue Bream easier to use? I propose making the initial > start up interface a simple content management system, kind of a Plone > light. This seems like a bad idea. Blue Bream attempts to be a development framework, not a content management system. You are suggesting we make an apple look like an orange, but still taste like an apple. Perhaps someone will build a CMS on top of Blue Bream, and perhaps this will make BB easier for people to learn, but that would need to be a separate project with separate goals. The experience from Zope 2 and Plone is that muddying the waters between an application and a framework makes a system harder for people to understand and use, not easier. > I know Plone just had a new release . I am not sure what all is in it, > but it would be great if Zope 3 on startup looked just like Plone, as > far as possible. Right out of the box allow users to register > themselves, and add in the stock content types. Calendar item, news > item, things like that, not the developer centric ZPT and DTML file. > Also allow the developer stuff, but under a different branch of a > hierarchical menu system.Maybe only people with a developer or > manager permission see those kinds of items. > It sounds like you're talking about through-the-web development, which is an idea now thoroughly discredited. TTW customisation has its place. Writing code through a web browser is just painful and brings long term pain once you grow out of tinkering mode and start having to worry about things like deployment and dev/test/prod lifecycles. > That is all I want. A simple content management system out of the box, > make it easy to add my own content type. Sure the serious hackers can > throw all that out, but for the average end user/developer, that would > be a hugely tempting improvement over using Plone. If you just want a simple CMS, I suggest you install a CMS. With Dexterity, by the way (available in Plone 4, probably standard in Plone 5), you can legitimately create content types through the web in Plone. > If I can add a few > dtml pages to display my content, it would all be very easy to get > started. Later I could export to the file system, and svn. > Please note that DTML is a dead (and horrid) technology. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZTK TTW Content Types Project Wiki
> > > Don't be too harsh on Grok/Dexterity. Dexterity has worked out how not to > repeat the definitions in interfaces, forms and content objects. It also > produces an application with suprisingly little redundant code. I urge you > to try it out. The benefits are of course quick turn around, version > control, testability and the debugger. > > Also note that (the Plone integration for) Dexterity allows you to create types entirely through-the-web. This can then be meaningfully transitioned to filesystem development without forcing you to start from scratch. The big, big problem with any kind of TTW development is that it usually breaks down badly when you move your code to a production server and then need to continue development in a separate environment. Code and configuration stored in a database is difficult to deploy and merge when the same database holds live data and content. The same problem exists where more than one developer needs to collaborate. Without source files and source control, it's impossible to give each developer their own sandbox. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Different Zope 3.4.2 Newbie feedback
On 9 September 2010 01:33, Christopher Lozinski wrote: > Here I am sharing my thoughts as a Zope 3.4.2 newbie. > As you have been told three times this week already: Zope 3 is in effect dead. You want to look at Grok (if you want less ZCML and more convention-based configuration) and/or Blue Bream (which is basically the evolution of Zope 3.4 under a new, less confusing name). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] lxml dependency in Zope 2.12.10 KGS
Hi, The Zope 2.12.10 KGS at http://download.zope.org/Zope2/index/2.12.10/versions.cfg specifies lxml = 2.2.6 There is no Python 2.6 Windows build for this egg, which means that this version cannot be installed on Windows under Python 2.6. Version 2.2.4 is the latest version with safe binary eggs for all platforms. What in Zope depends on lxml? Why did we pin to 2.2.6? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] lxml dependency in Zope 2.12.10 KGS
On 10 September 2010 14:04, Hanno Schlichting wrote: > Hi. > > On Fri, Sep 10, 2010 at 2:45 PM, Martin Aspeli > > > wrote: > > The Zope 2.12.10 KGS at > > http://download.zope.org/Zope2/index/2.12.10/versions.cfg specifies > > > > lxml = 2.2.6 > > > > There is no Python 2.6 Windows build for this egg, which means that this > > version cannot be installed on Windows under Python 2.6. Version 2.2.4 is > > the latest version with safe binary eggs for all platforms. > > This is unfortunate, but really a problem for the lxml community and > not us. So the lxml community cannot keep up with providing binary > Windows eggs. This cannot force us to stick with old and buggy > versions of the software. > Well... the problem, apparently, is that libxml2 doesn't (or didn't?) have suitable Windows binaries, or so I'm told. I'm also not sure the bug fixes in 2.2.5 onwards are very important in a Zope context, since they seem to deal with Python 3 compatibility mainly. > > What in Zope depends on lxml? Why did we pin to 2.2.6? > > 2.2.6 was the latest stable version available at the time of the > release. 2.2.7 had known problems with newer libxml2 versions, but I > see there's a 2.2.8 out now, which we should update to. > > I'm not sure about the actual dependency situation. I think it's more > or less a convenience pin, as lxml is used very often in Zope related > projects. We provide a known good version for it in the Zope Toolkit > KGS for the same reason. > This sounds wrong to me. If we *are* going to use a convenience pin, then surely the ability to install on the world's most-used operating system has to be part of the convenience. ;-) If we don't use it, we shouldn't pin it, IMHO. We found this problem because the Zope KGS was overriding another KGS where we had pinned lxml to 2.2.4. I don't think Zope has any business getting in the way of that. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] lxml dependency in Zope 2.12.10 KGS
On 10 September 2010 14:26, Hanno Schlichting wrote: > On Fri, Sep 10, 2010 at 3:17 PM, Martin Aspeli > > > wrote: > > If we *are* going to use a convenience pin, then surely the ability to > > install on the world's most-used operating system has to be part of the > > convenience. ;-) > > That's a lame argument. Windows is almost irrelevant for the market we > are in - web server deployments. Erm, you think so? Maybe we should do a poll on how many Zope / Plone developers use Windows on the desktop. Or look at how many people download the Windows installer. You need a dev environment, not just deployment, and a lot of people are on Windows. > Our own community is barely able to > keep up providing the most basic Windows support and ensuring tests > pass. As long as we don't have more community volunteers actually > caring about Windows support, I won't let it be an argument to > penalize the rest of the community. > When the software breaks, people go elsewhere. I didn't say Windows support was easy, or any fun. But we have to decide: do we care about people who have made (or are forced to make) different technology choices than us, or do we tell them their platform is unsupported? > > If we don't use it, we shouldn't pin it, IMHO. We found this problem > because > > the Zope KGS was overriding another KGS where we had pinned lxml to > 2.2.4. I > > don't think Zope has any business getting in the way of that. > > The KGS is a base KGS you can use. Nobody forces you to stick to it. > In fact for every single deployment of your own you will need to > extend it. I don't see a problem with the few people using Windows and > not installing compilers on their platforms to change one version pin. > I think you're missing the point: - We shouldn't pin software we don't use. It may be well intentioned, but if we don't depend on it, we shouldn't take responsibility for it, or give the perception that we take that responsibility. - If we do depend on it, we need to make sure it works on the platforms we support. QA isn't something you do only when it's easy to do in your local dev sandbox. - If we suddenly no longer support Windows, we better have the guts to come out and say it, stop producing Windows eggs for Zope 2 stuff and explicitly state that people cannot and should not use Windows for Zope development. I hope that's not the case, though. ;) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] z3c.form buildout broken
On 21 November 2010 16:16, Roger wrote: > Why do we use such crapy parts like omplette in z3c.form? I never do any development whatsoever without collective.recipe.omelette. It may not be right for z3c.form, but it's not crappy. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZPublisher: using zope.formlib and z3c.form in Zope 2
Hi, On 2 March 2011 08:43, yuppie wrote: > Hi! > > > ZPublisher.Publish and zope.publisher.publish process form inputs > differently. Zope 2 returns encoded strings unchanged if no converters > are specified. zope.publisher converts encoded strings to unicode. > > One major reason why zope.formlib and z3c.form can't be used directly in > Zope 2 is the fact they expect decoded form values. five.formlib uses > special base classes and plone.z3cform monkey patches the base classes > in z3c.form. > > > Proposal: > > - Products.Five.browser.decode should be moved to ZPublisher. > processInputs and setPageEncoding are publisher related code. +1 > - After traversal and before calling the object > ZPublisher.Publish.publish should figure out if the object expects > zope.publisher behavior. Either we use a new interface for this or we > use zope.publisher.interfaces.browser.IBrowserPage: AFAICS in Zope 2 > land only zope.formlib and z3c.form based views implement IBrowserPage. Isn't this in zope.browserpage now? > - If the object implements that interface, the request is post processed > using processInputs and setPageEncoding. +1 > - plone.z3cform uses a modified version of processInputs and doesn't use > setPageEncoding. Can anybody explain why? I guess that are no z3c.form > specific reasons. Maybe the changes can be merged back to Zope? processInputs() in Five was very buggy. I thought I'd merged those changes into Zope 2? I don't know what setPageEncoding() does, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Anyone want to do Google Summer of code mentoring for PSF?
On 20 March 2011 13:46, Jim Fulton wrote: > I think we ought to come up with a much cleaner way of defining > default configuration. (Pyramid does this by passing default values in > adapter calls, but I think we can do a lot better than that.) I'd > like to see us come up with a "pythonic" way to wire components up > that can be overridden through registration (through zcml or > otherwise). Ideally, the mechanism shouldn't "feel" like > "configuration" but like "programming". You mean like http://pypi.python.org/pypi/grokcore.component http://pypi.python.org/pypi/grokcore.view http://pypi.python.org/pypi/grokcore.viewlet http://pypi.python.org/pypi/grokcore.security ? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Anyone want to do Google Summer of code mentoring for PSF?
Hi,
On 20 March 2011 15:00, Hanno Schlichting wrote:
> On Sun, Mar 20, 2011 at 3:28 PM, Jim Fulton wrote:
>> - The mechanism shouldn't require something to "grok"/analyze the
>> code. The mechanism should be explicit. This is implied by
>> "pythonic". I remember Grok being more implicit than skimming the
>> links above suggest. Perhaps Grok has has become more explicit than
>> I remember.
>
> Both Grok and Pyramid (or martian and venusian really) do a scan of
> the code to find the registration hints.
>
> I think you cannot avoid this, if you want to support an explicit
> configuration phase. Otherwise the first import of a module could
> occur at any point at runtime and have a configuration side-effect
> like registering a new view. Personally I find the venusian approach
> pretty simple and explicit enough.
>
>> Note that the move from "advice-based" syntax to class decorators
>> provides a good opportunity to revisit how some of this works based
>> on experience using the ztk.
>
> Taking one of the examples of grokcore.component, I think there's a
> lot that can be made simpler:
>
> import grokcore.component
> from zope.i18n.interfaces import ITranslationDomain
>
> class HelloWorldTranslationDomain(grokcore.component.GlobalUtility):
> grokcore.component.implements(ITranslationDomain)
> grokcore.component.name('helloworld')
>
> Based on my Pyramid exposure, I'd write this as something like:
>
> from something import utility
> from zope.i18n.interfaces import ITranslationDomain
>
> @utility(ITranslationDomain, name='helloworld')
> class HelloWorldTranslationDomain(object):
> pass
>
> This does mean hardcoding default values into the registration calls.
> I'm not sure I see the value of avoiding this, as long as there's a
> way to unregister this utility and re-register the class with other
> values.
I agree - this is nicer. In Grok's defence, class decorators didn't
exist when martian was conceived. :)
I do wonder whether it's nicer *though*, though, to justify yet more
proliferation of configuration syntax.
I appreciate that in Python 3 the in-class advice (which was pioneered
by zope.component/zope.interface, don't forget) may not work properly,
so we may not have any choice eventually. But saving two lines of code
in a hypothetical example doesn't necessarily outweigh the confusion
that comes from having multiple ways to do things, and grokcore.*
configuration is in use in the wild outside of Grok.
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Anyone want to do Google Summer of code mentoring for PSF?
Hi, On 20 March 2011 15:29, Jim Fulton wrote: >> I think you cannot avoid this, if you want to support an explicit >> configuration phase. Otherwise the first import of a module could >> occur at any point at runtime and have a configuration side-effect >> like registering a new view. Personally I find the venusian approach >> pretty simple and explicit enough. > > I disagree. First, the notion that you'd import at run time is pretty odd, > unless you count start-up in "runtime". I think Hanno was saying that configuration at import time is unpredictable and leads to ordering problems and "circular import" type risks. This shouldn't really be a surprise to anyone. > Second, well, really, I'm not ready to give up on a straightforward > definition of wiring that doesn't rely on module scanning. I recall Chris suggesting some improvements of the zope.configuration API to make it user friendly. Right now, it only really makes sense in the context of writing ZCML directives. Pyramid has largely done that work, though I'm not sure how "compatible" it is at this point in time. With a cleaner, better documented zope.configuration API, you can do configuration in your "__main__" function or whatever works on startup. One option then is to use the indirection of ZCML or the indirection of martian/venusian style scanning, which may make sense for frameworks and pluggable apps, but less sense for more closed systems. Then again, it feels like Pyramid has largely done this already. ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hi, On 25 March 2011 13:17, Jim Fulton wrote: > On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring wrote: >> Hello Uli, >> >> I've spent quite some time thinking (and partly coding) about the same >> issues you mention (but didn't feel ready to talk about it here, yet), >> so I'm glad that maybe we can start thinking about them together >> >> I think your email addresses two quite different topics, so I'll split >> my reply. First up: test support for zope.component. Second part: >> about the concept of test layers. >> >> * Uli Fouquet [2011-03-24 01:05]: >>> Right now we have a problem with pytest integration when it comes to >>> ZCA setups [...] all tests share the same global ZCA registrations >>> and changes to the registrations in one test will affect other tests >>> run thereafter. We have a lack of test isolation. >> >> Exactly. This issue has bitten me too in various places, and as far as >> I know there are no solutions for it, yet. > > The classic solution is to start tests with empty registries, or, if > you're using layers, with some baseline registries. plone.testing (which is Plone non-specific and will shortly be BSD licensed) allows for stacking of ZCA registries. It has to do some ugly hacking to achieve this, since zope.component stores handles to the global registry in *three* different modules and update them in weird ways depending on the registry hooking, but it's well tested and robust now. See http://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.py and http://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.txt. >> What I envision to solve this issue is that test support for >> zope.component should work the same way as with the ZODB. There, we >> have a *stack* of Databases (DemoStorages, to be precise) that wrap >> each other, where each one delegates reads downwards, but keeps writes >> for itself. So you might have one database for the layer that provides >> the baseline, and each test (in its setUp) gets its own database where >> it can do whatever it wants, because it is thrown away in its >> tearDown. >> >> In principle, quite a few of the mechanics to do the same thing with >> zope.component registries are already there (since a registry keeps a >> list of base registries it will delegate to when something can not be >> found in itself). And as Hanno and Godefroid mentioned, plone.testing >> does something in this direction already. (And, it bears repeating, >> in its core has no dependencies on Plone or Zope2.) > > I like the idea of stacking registries. plone.testing implements this. If we could fix zope.component to make the implementation less ugly, that'd be a big win. >> But as far as I see, there are issues that plone.testing does not >> address: >> >> 1. I've been going over this stuff with my colleague Thomas Lotze, and >> we realized that just squeezing in a new registry and bending its >> bases to the previously active one is not enough for full isolation, >> since this does not cover *deleting* registrations (one, you can only >> delete the registration from the precise registry it was created in, >> and two, in the just-bend-the-bases approach, once you delete a >> registration, it's gone forever). Correct, although this is in practice extremely rare. I'd say it's much better to control setup more carefully and not have to "undo" in a child layer. >> I think to provide full isolation, we need to make *copies*. And since >> zope.component in general supports a chain of registries, we probably >> need to make copies of each of them. Copying is very hard. It was my first attempt in plone.testing and didn't work out well. You need to support pickling of registries for local/persistent component registries. I cannot begin to tell you how many weird pickling errors I found and had to work around. > Is deleting registrations important? This seems like an odd use case. > If it's needed, I would suggest starting with a baseline (e.g. stack) > that doesn't include the component you want to test deleting, then > adding in setup. +1 >> 2. zope.component has two entry points, the global site registry and >> the current registry (getGlobalSiteManager and getSiteManager). >> The current registry can be anything, or more precisely, you can call >> getSiteManager.sethook(callable) and provide a callable that returns >> the current registry. >> >> I think to provide test support for zope.component (i. e. generally, >> at the "library level"), we need to support both entry points. > > Why? Why would someone care about anything other than the current > effective configuration. Agree. There is a problem in that provideAdapter() and friends don't use getSiteManager() - the always use the global site manager. And there are parts of zope.component that use module level variables directly, ignoring hooks. >> The >> global one is not hard, but the getSiteManager one gets nasty really >> fast, because of course we have t
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hi Jim, On 25 March 2011 14:12, Jim Fulton wrote: >> Agree. There is a problem in that provideAdapter() and friends don't >> use getSiteManager() - the always use the global site manager. And >> there are parts of zope.component that use module level variables >> directly, ignoring hooks. > > These are meant to work this way. > > If you want to do local configuration, you should explictly select the > relevent registry/manager or call getSiteManager and use the result. I sortof understand, but it makes it impossible to let people use provideAdapter() & co in test setup and still retain some kind of layered test setup, without the kind of hacks we do in plone.testing. -- but anyone could at any time call getSiteManager.sethook to change it! >>> >>> Seriously? Nobody calls that but deep infrastructure code. >> >> People do call zope.site.hooks.setHooks() sometimes, though, e.g. upon >> traversal. > > This was never meant to be an application-level feature. I find the > notion that people would call these dureing traversal to be > disturbing. Are you sure you're not confusing this with setSite? Sorry, I meant setSite() above yes. Although sometimes people call setHooks() and then setSite(site) in test setup, because setSite() doesn't work until setHooks() has been called once. I think this may sometimes just be cargo-cult, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation
Hi, On 26 March 2011 08:11, Wolfgang Schnerring wrote: > Hello, > > * Martin Aspeli [2011-03-25 13:58]: >> plone.testing (which is Plone non-specific and will shortly be BSD >> licensed) allows for stacking of ZCA registries. >> [...] >> Again, plone.testing is the result of hours and hours of finding weird >> problems, so I'd recommend you don't discard the knowledge there. I >> think a best case scenario would be for plone.testing.zca to just be a >> delegate for something more robust in zope.component.testing. > > I wasn't aware plone.testing had something for the ZCA, before I read > that in this thread yesterday. I'm glad to hear that, and I certainly > don't want to discard anything. But from reading the code and what you > wrote, I get the impression that plone.testing does not yet solve the > issue completely, precisely because of the two points I brought up and > that continue below: It certainly doesn't solve the "unregister" (non-?)use case. It has the advantage of being in use in the wild today, though. :-) >> On 25 March 2011 13:17, Jim Fulton wrote: >> > On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring >> > wrote: >> > > we realized that just squeezing in a new registry and bending its >> > > bases to the previously active one is not enough for full isolation, >> > > since this does not cover *deleting* registrations >> > >> > Is deleting registrations important? This seems like an odd use case. >> > If it's needed, I would suggest starting with a baseline (e.g. stack) >> > that doesn't include the component you want to test deleting, then >> > adding in setup. > > I've wanted to specifically nuke registrations sometimes, the pattern > being, load the package's configure.zcml in the layer, and then delete > something (to demonstrate some error handling behaviour). > > I agree that this use case is rare, but I'm not sure it is rare enough > that we should ignore it. I need to think about this some more, > though. It would be better to explicitly register what you need. I think you either need to consider a package's ZCML-loaded configuration as an atomic part of test setup, or you need to break it down into individual registrations. In the first case, your test fixture is "package foo's configuration is loaded", which is of course valid. In the second case, your fixture is "the following components, some of which happen to be in package foo, are registered". I don't think a fixture of "package foo's configuration except component X and Y" is all that useful. Of course, there may be cases where an unregister is needed, but this is probably something you should solve locally. For example, you can explicitly unregister the component at the beginning of your test and reinstate it at the end of your test, or indeed in layer setup and tear-down. Automating this with stacking is not a great win. Please also take my word for it when I say that copying the whole registry is non-trivial and would rely on brittle ZCA internals. I tried. :) >> > > 2. zope.component has two entry points, the global site registry and >> > > the current registry (getGlobalSiteManager and getSiteManager). >> > > The current registry can be anything, or more precisely, you can call >> > > getSiteManager.sethook(callable) and provide a callable that returns >> > > the current registry. >> > > >> > > I think to provide test support for zope.component (i. e. generally, >> > > at the "library level"), we need to support both entry points. >> > >> > Why? Why would someone care about anything other than the current >> > effective configuration. > > Because that's the API that zope.component offers, it conceptually > deals with *two* registries: the one you get from getSiteManager and > the one you get from getGlobalSiteManager. > > I agree that it is unlikely that client code will *read* registrations > using getGlobalSiteManager -- since most everyone uses > zope.component.getUtility etc. (which in turn uses getSiteManager). > But *writing* is a different matter. Just one example: > zope.component.provideUtility etc. uses getGlobalSiteManager, while > the ZCML directives use getSiteManager. > > At least as long as zope.component.provide* uses getGlobalSiteManager > instead of getSiteManager, I maintain that test infrastructure needs to > 1. wrap the global registry > 2. wrap whatever getSiteManager currently returns > (Otherwise we might be able get away with not doing (1), but I think > we'll always need
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi, On 26 March 2011 14:18, Wolfgang Schnerring wrote: > Because, while test layers are nice because they have the above > properties, I'm not too happy with the current implementation, namely > the use (or is it abuse? ;-) of __bases__ and __name__, which leads > very naturally to not-so-helpful implementation patterns. As a > concrete example, some of the test layers of ZTK packages, most > notably the successors of zope.app.testing.functional in > zope.component, zope.app.appsetup, zope.app.wsgi, are implemented > right now in a way that does not have properties 2+3. > > The most straightforward way to improve that situation would be to use > plone.testing, since that provides a layer implementation that is both > sane and easy to use, and has all the nice properties. (Maybe even > fold some of it into zope.testrunner itself.) I'd be happy to move layer.py from plone.testing to zope.testrunner for sure. > But then I realized, that's a major undertaking, so we better think > about what the actual goals are before changing lots of stuff. And > then I heard people expressing interest in other test runners. > (py.test and nose seem to be the biggest players, stdlib's > unittest/unittest2 seems to be woefully behind in terms of test > selection and other features like coverage or debugging. I for one > happen to like zope.testrunner, but if there's a good alternative, why > not combine our efforts?) > I also found that it is rather hard to explain how to use test layers > in a way so you actually get the above properties, so I started > wondering whether there was a cleaner way to get them. Personally, I rely a lot on the layer mechanism and would not want to lose it. > So I guess I have three questions: > > 1. (The starting point:) How can we improve the test infrastructure > situation of the ZTK packages? I'd say having more reusable test layers would help. > 2. (The main thing:) Are test layers, both the concept and the > implementation as epitomized by plone.testing, the way to go or is > there a cleaner alternative? When we looked at how to improve Plone's testing story, we couldn't find anything better. One thing I would like is a better way to support "python setup.py test". > 3. (But only tangentially, I don't want to sidetrack the discussion at > hand nor start a flamewar:) Is there a compelling alternative to > zope.testrunner that would make it worthwile to think about either > generalizing the fixture support or even migrating away from > zope.testrunner? It's a good question to ask, though I think this would be a really big undertaking for maybe only marginal gain (if any). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi,
On 27 March 2011 15:54, Uli Fouquet wrote:
> The (limited) experiences with py.test, however, were awesome. Some
> points that are quite cool IMHO:
>
> - Easy finding of tests: just write some ``test_function`` in a
> ``test_module`` and it will be found and executed. That also makes
> py.test tests more readable and maybe more intuitive.
I'm not sure this is always a good idea. It feels a bit implicit, and
having a base class isn't really a big problem, IMHO. It seems a bit
like the kind of thing that sounds cool (look, it's even easier!), but
in practice makes little difference.
> - Lots of setup code (unrelated to fixtures) can simply be skipped. No
> need to do the ``testsuite = `` over and
> over again. Maybe the main point of py.test.
You don't need that for zope.testrunner either, of course, at least
not when using unittest base classes.
> - py.test is more widespread in the Python community (that's my
> impression; I can't proof it)
What about nose?
> - Support of unittest/unittest2: you can write standard lib setups
> (defining TestCases; no need to also write testsuite-setup stuff) and
> they will be found/executed. zope.testrunner for instance does not
> support the new `setUpClass`/`tearDownClass` concept of unittest2
> (yes, you would use layers in that case; but it might be nice if
> zope.testrunner would support also class-wide fixtures in
> unittest2-style; people from other worlds might expect that to work).
zope.testing should definitely gain support for the new unittest2
hooks. That wouldn't be very hard, though. ;-)
> Main drawbacks I see on py.test side are:
>
> - Lack of layer support (yet). Maybe we can do something about that in
> `zope.pytest` based on `plone.testing.layer`.
>
> - Limited doctest support. It is quite difficult (AFAIK) to define
> fixtures for doctests or to even set the usual doctest options
> (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests
> are simply collected and executed and not much finetuning is possible.
With zope.testrunner, you *do* need a test_suite method to run
doctests. I think that's a good thing. Look at plone.testing's README
for examples.
> My very own concern about the latter point is: if this cannot be solved
> satisfactorily (in terms of readability and ease of use at least),
> py.test will not become a really hot candidate in the testing framework
> game on the Zope side at all. There are too many doctests used already
> and whether you like them or not: every testing framework used should
> offer at least minimal support for doctest fixtures and doctest options.
> But I might just have missed these features in py.test and they are
> provided already.
FWIW, I think we should stop using .txt doctests for unit tests.
Doctests should be used to test *documentation* ("the examples are
valid"). For actual unit tests, writing tests in a unittest class is
almost always better in the long run. doctests don't scale well and
discourage the kind of ad-hoc "this seems broken, I'll just write a
quick test" or "I just fixed a bug, better add a regression test"
testing.
> For now I think that there is absolutely no need to think about a
> general move to py.test for the ztk.
I think there's benefit in unifying the concepts and support for
concepts like layers so that people can use the test runner they
prefer.
Maritn
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts
On 28 March 2011 15:45, Tres Seaver wrote: > The vast majority of the doctest testcases in zope.* packages fall into > this category: poor isolation, lots of edge cases which would obscure > any real narrative docs, of which there are almost none. I believe the > conflict is intrinsic, here, and not just an accident of careless / > naive implementation: exercising all the preconditions of the contract > of the function-under-test makes for really poor documentation, but is > essential to good testing. Agree. Here's what I've found and learned the hard way: doctests are sometimes easier / more fun to write the *first* time I write tests. It's easy to just start writing, and the narrative helps me think about what I'm doing. Plus, it feels like I'm saving time on writing docs. They are much worse all subsequent times. Maintenance becomes a soup. Quick tests for edge cases feel like they obscure the narrative, so I may forgo them. Refactoring is painful. Tool support is poorer, e.g. no stepping through with pdb and no pyflakes. And people find the docs underwhelming. If I'm doing it wrong, I'd like to see it done "right". Manuel is kind of cool, but I'm not sure it really addresses the issue. It's a better doctest, not a better unittest. Most zope.component packages only get away with simple doctests because they are simple/small packages (which is a good thing, mainly, of course). One of the main objections to unittest(2) seems to be that it's "Javaesque". Yes, JUnit was the first library to solidify many of the current, cross-language testing patterns. We shouldn't be so NIH-stricken that we can't see the benefit of being consistent with patterns used in most other languages and test frameworks these days. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation
Hi, On 4 April 2011 17:30, Wolfgang Schnerring wrote: > So, how can we proceed here? Should I (and Thomas) try to get a > proof-of-concept implementation of this based on plone.testing? Or should > we think about what it takes to merge most of plone.testing's ZCA > support into zope.component itself first? I think either approach is valuable, and not necessarily mutually exclusive. I do care about the plone.testing API, which is used in production, so bear that in mind. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZEO Server service installation on Windows
Hi, I just found out that, since Zope 2.12 (or 2.13?) it's no longer possible to install a ZEO server as a Windows service. Previously, it used to be possible to do: > bin\zeoservice install (where bin\zeoservice is installed by plone.recipe.zeoserver), but apparently no longer. Is this really the case? It seems a pretty serious omission in our Windows story, and all the more puzzling since zopectl (and the instance scripts installed by plone.recipe.zope2instance) has an "install" option that still works. Has anyone got a solution, or at least is willing to chase this down? Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] component registry navelgazing
On 12 June 2011 21:48, Chris McDonough wrote: > Currently if you ask a registry to singly-adapt an object to an > interface, and the object you're trying to adapt implements that > interface, here's what happens: > from zope.component.registry import Components c = Components() from zope.interface import Interface, implements class IFoo(Interface): pass > ... class Foo(object): > ... implements(IFoo) > ... foo = Foo() c.queryAdapter(IFoo, foo) > > > In order to get the object itself back from such an adaptation, you need > to use the default= argument. > c.queryAdapter(IFoo, foo, default=foo) > <__main__.Foo object at 0x24a3910> > > This seems slightly inconsistent with the adaptation worldview imposed > by getAdapter/queryAdapter. I think it would be more consistent if > "c.queryAdapter(IFoo, foo)" returned foo if foo already implemented IFoo > and there was no other more specific adapter registered for the IFoo/foo > pair in the registry, no? +1 Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Sharing session between different zope servers
Hi, You can use collective.beaker to manage your sessions with beaker, and store on the filesystem (if all on the same server) or memcached (if on different servers). That's a code change, though. Martin On 15 June 2011 17:50, Subish K S wrote: > Hi, > > We have 10+ Zope (version 2.7) servers ( on different machines). Right now > each Zope server independently handles session management which means that if > a Zope server fails the users on that Zope server do not get redirected to a > functioning Zope server but get "stuck" on the failed Zope server. > Is there any way to manage the session of all the zopes servers in a common > place. > > Thanks in advance... > > Regards, > Subish > ___ > Zope-Dev maillist - Zope-Dev@zope.org > https://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > https://mail.zope.org/mailman/listinfo/zope-announce > https://mail.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 3 July 2011 16:44, Hanno Schlichting wrote: > On Sun, Jul 3, 2011 at 7:09 AM, Chris McDonough wrote: > > Zope still needs to the virtual host monster (or something like it) even > > with the WSGI publisher; there's nothing equivalent in the WSGI world > > (unless you could repoze.vhm, which is essentially just the virtual host > > monster, and probably doesn't need to live in middleware; no one uses it > > except people who use repoze.zope2). > > I'm expecting us to use repoze.vhm. But I've left the VHM in the code, > so it's easy to install one if you still need it. For some time I > expect Plone to install a VHM as part of its installation process. > > > I don't have any skin in this game, but FTR, Mike Bayer isn't feeling > > all that confident about Beaker's sessioning component (or so he has > > told me). Beaker was originally made as a caching component, and had > > sessioning jammed into it quite late; nobody is really maintaining the > > sessioning component of it now. > > Well, if I can choose between modern unmaintained code from Mike Bayer > and stone-age unmaintained code from Zope, it's still an easy choice > ;-) > > And looking at the basics of what Beaker does here, it's still much > more useful and of better quality than what we have in Zope 2. > > If there's any other non-framework-specific session machinery out > there, we could use that as well. But I think most other stuff is tied > into Django. FWIW, we have a high-performance, high-load application in production on Plone 4 with collective.beaker relying heavily on sessions, and I'm not aware of any problems with it. We use the memcached backend across two physical servers and a large number of Zope clients. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
Something of a meta-comment on this thread: It sounds like people are broadly in agreement on the direction, but not communicating enough about what's actually going on. I think it would be useful to keep some kind of roadmap wiki on zope.org, or at least post to the list periodically saying, "this is what we're about to start doing". Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 09:42, Hanno Schlichting wrote: > What you are describing is exactly what I meant by old legacy Zope2 > applications. > > You should be able to use this style of development with Zope 2.13. > But you won't be able to upgrade to newer versions of Zope 2 anymore > and expect your code to work unmodified. We discouraged this style of > development for the past six years. It should come as no surprise that > it will come to an end at some point. > I would've thought it would also be possible for those who rely on this to maintain the relevant eggs as optional installations against Zope 2.x, no? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 10:18, Hanno Schlichting wrote: > On Tue, Jul 5, 2011 at 11:03 AM, Martin Aspeli > wrote: > > I would've thought it would also be possible for those who rely on this > to > > maintain the relevant eggs as optional installations against Zope 2.x, > no? > > The ZMI is not a package - we don't have a split into zope and > zope.app in Zope2. Once there's no more ZMI, Products.PageTemplates > stops using RestrictedPython and the OFS base classes don't inherit > from Acquisition.Implicit anymore, it'll be really hard to keep the > legacy development approach working. > I think it might be useful to spell out the reasons behind this (here, or better yet, somewhere more permanent like zope.org). I can imagine people reading this and wondering why it's a good idea, especially those who have an investment in the existing technologies. > Someone might try, but I think it's not a wise decision to spent any > resources that way. At some point every application written in the > legacy style has to be rewritten. I think it would be a better use of > resources for anyone to start doing that than maintaining a dead-end. This is a pretty sweeping statement that I think could cause a lot of nervousness. It might be the right thing in many ways, but we need to at least provide a bit more context. If you're a business that's invested dozens of person-years into a product, the prospect of rewriting could seem fairly daunting. At least we, as the Zope 2 community, need to set out the case for change and some kind of idea of timing and transition path, even if that means in some cases getting to a "long term maintenance" release and in other cases evolving away from certain technologies whilst being confident to keep using others. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 10:31, Hanno Schlichting wrote: > On Mon, Jul 4, 2011 at 12:19 PM, yuppie wrote: > > Long-term maintenance for Zope 2.13 would give these > > projects/deployments at least a few more years. > > Yes. I'm willing to cut releases for it for quite a while. I just > expect to see active maintenance from the Plone community to stop in a > year or two. Judging from the ongoing maintenance we currently have > for Zope 2.10 or 2.11 I don't think it's very realistic to expect much > to happen once the Plone guys stop. > > >> What I'm outlining here has of course almost nothing to do with the > >> original idea and scope of Zope 2. Maybe at some point this should get > >> a different name ;-) > > > > I don't want to discuss names, but I think the next release from Zope > > trunk should be explicitly a new *major* release. > > I think that's perfectly fine. Since I broke backwards compatibility > with a number of changes I did, a major version increase is warranted. > > So we just got ourselves a Zope2 version 3.0. And no, naming it 4.0 or > 5.0 or anything else doesn't make it any better at all. So 3.0 is the > most sensible one :) Boy, that's going to be confusing. :) I'd actually favour calling it Zope2 4.0 just to avoid any mix-up with the defunct Zope 3, although I don't think there are any particularly good options here. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
On 5 July 2011 11:10, Jens Vagelpohl wrote: > On 7/5/11 11:56 , Martin Aspeli wrote: > > On 5 July 2011 10:31, Hanno Schlichting wrote: > >> So we just got ourselves a Zope2 version 3.0. And no, naming it 4.0 or > >> 5.0 or anything else doesn't make it any better at all. So 3.0 is the > >> most sensible one :) > > > > > > Boy, that's going to be confusing. :) > > > > I'd actually favour calling it Zope2 4.0 just to avoid any mix-up with > the > > defunct Zope 3, although I don't think there are any particularly good > > options here. > > I actually think it's a brilliant idea to "skip" 3.0 and call it 4.0. > > As Martin said, the potential for confusion is very high. A 4.0 would > not only steer around confusing Zope3 3.x and Zope2 3.x, it would also > make it easier to move back to the simple "Zope" moniker without any > qualifying number tacked on. People who only look at version numbers > would now choose Zope 4.0 instead of falling into the "unmaintained" > Zope3 trap. I would tend to agree, given that we now have Blue Bream. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] direction
Hi, On 5 July 2011 11:26, Tobias Helfrich wrote: > > Hi Hanno > > > On Tue, Jul 5, 2011 at 10:19 AM, Tobias Helfrich > > wrote: > > > OK, so you do think that we might use Zope 2.12 for a quite > > long time > > > without thinking about anymore updates? Will there be any security > > > updates for Zope 2.12 in the future? > > > > You want to use Zope 2.13. 2.12 is at the end of its active > > maintenance cycle as is Python 2.6 (the only Python version > > it supports). > > > > Zope 2.13 brings support for Python 2.7, which is a long-term > > maintenance release. > > OK, thx for the info. So we will be able to use Zope 2.13 with the > techniques mentioned before? That will give us another two years to > think about going on with different styles. So basically Zope 2 will > be the framework for Plone only, because the community which is/was > using Zope 2 for standalone individual projects has decreased to nearly > none. I think it would be very sad if that happened, especially since there evidently demand from other projects. What I think is clear is that to evolve Zope 2, we need to shed some baggage and make some deeper changes to allow us to achieve some of our goals (e.g. WSGI, simplified stack, simpler and more easily controllable security, less magical traversal, more comprehensible publisher). Plone is obviously a big consumer of Zope 2 and I would expect Plone to have a major influence on Zope 2's evolution. But ERP5 is another big consumer, and we shouldn't ignore that. > So it might be a good idea to look for something completely > different? I don't think that Plone will be able to do everything we > want it to. Or do you think we can stick with Zope 2 but change the way > of building applications to ZPT/TAL? So we have to get rid of all DTML > and what about an alternative to the ZSQL Methods? I think keeping DTML as an optional installation should be quite feasible. It's just possibly not something that the core Zope 2 team want to maintain anymore in the standard distribution. ZSQL Methods may be a similar story, in fact, especially as they rely on DTML. However, I'd encourage you to look at SQLAlchemy, which is way nicer to work with. > OK, i have subscribed to the mailing list today, so unfortunately > i haven't found this sort of information anywhere else. I don't want > to blame you or anyone else for that, but it's not nice to hear that > right now :-( I think there is some characteristic bluntness in Hanno's emails, but please realise that none of this is going to happen over night, and existing codebases are not going to magically disappear. Sometimes we have to be a bit more radical to understand the art of the possible and build a future platform that will support future needs. That doesn't mean there can't be both migration paths and long-term stable versions. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] www.zope.org relaunched
On 7 July 2011 12:58, Andreas Jung wrote: > Dear Zope Community, > > on behalf of the Zope Foundation I please to announce > the relaunch of the new www.zope.org web site. > > http://www.zope.org > > The "old" zope.org site will available for the time being > in (reduced form) under > > http://old.zope.org > > Many thanks to my team: > > - Kai Mertens > - Michael Haubenwaller > - Jens Vagelpohl > - Johannes Raggam > > Andreas Jung > Zope Foundation Fantastic work - thanks so much for succeeding where so many (myself included) have previously failed. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposal for merging jbohman-zope.registry branch of zope.component
Hi, On 17 August 2011 03:50, Tres Seaver wrote: > - - Land 'zope.registry' as a full ZTK package, with its own Launchpad > artifacts, etc. This step may also involve moving bugs from > zope.component to zope.registry. This is not a major issue, but just be aware that there's a widely-used package plone.registry (which, in fact, has no dependencies beyond the ZTK) that serves a rather different purpose (http://pypi.python.org/pypi/plone.registry). It may be a bit confusing to people if we have a zope.registry that means something else, so perhaps we could call it something else? As I said, not a major concern. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Revert removal of ++skin++ in Zope4?
On 16 November 2011 11:30, Christian Theune wrote: > Going down into the new ZMI project I find it to be the most > light-weight approach without adding an extra dependency. What is this project? ;-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Enabling External Methods in skin folders
On 17 November 2011 11:28, wrote: > Hi, > > I have a bunch of External Methods I'd like to make available in a skin > form, and which reload in the same way a page template would if it was > modified and the server was in debug mode. External methods should not require restarts either. > What's the recommended product for enabling this now? A more robust approach may be to turn your external methods into views, utility functions called from other views, portal_setup upgrade steps, or whatever other purpose they serve. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 release management
Hi, On 17 November 2011 12:25, Laurence Rowe wrote: > Along with David Glick, I would like to volunteer for the Zope 4 > release management role, where I would take responsibility for > producing the initial release of Zope 4 and David would then take over > for the maintenance releases. w00t :-) +1 from me > In doing so, I thought it would be helpful to set out our > understanding of the mission of Zope 4. If accepted I'll work to > produce a first draft of a roadmap based on the outcome of the recent > sprints and discussions on this mailing list. > > > Mission > --- > > Zope 4 will be the first of several releases that seek to simplify our > software stack. Over the years Zope 2 grew through the adoption of new > technologies, notably Zope 3, but rarely removed older ways of doing > things. Below, 'Zope 4' refers to the series of releases including > Zope 5, 6, etc. > > Over the series of releases, Zope 4 will progressively remove more and > more software as we transition to using software components shared > with other Python web frameworks. > > It is as important to state what Zope 4 *is not* as what it should be: > > * Zope 4 will not seek to be of interest to those developing new > applications. They should instead look to projects such as Pyramid > that build on the experience and technologies of Zope without regard > for the backwards compatibility concerns that will constrain Zope 4. > > * Zope 4 will not seek to innovate in itself but encourage innovation > in software components shared with the wider Python web community. > > * Zope 4 will not move so quickly that it becomes impossible to make > Plone, its main consumer, work with it. > > * But neither will Zope 4 seek to retain backwards compatibility at > any cost. As the basis of Plone 4, Zope 2.13 will see maintenance > releases as long as Plone 4 is supported. > > * Zope 4 will not have a release cycle independent of Plone. Zope 4 > only exists as a transitional path for Zope 2 based applications and > experience has shown that Zope 2 releases not used in any Plone > release do not receive the same level of ongoing maintenance. I think these are sensible principles, but we shouldn't disregard the Zope community outside Plone. Hence, if Zenoss, Silva, ERP5 and others are willing to contribute and maintain code, they should not feel shunted out of the development process. That's not to say we should succumb to indefinite maintenance of all components on the odd chance iet may be needed by "someone" (we'll have a stable Zope 2 branch for that), but I believe we're stronger as a bigger community with more voices than as a narrow group with only one goal. > Development Process > --- > > We want to encourage all features to be developed on separate feature > branches so we can maintain a stable trunk. Before these feature > branches are merged they should be posted to the mailing list for > review. > > This process will necessitate a lot of merging, so I want to propose > that we move to Git for development (something we found very helpful > at our recent San Francisco Zope 4 sprint.) I don't have any opinion > on where the canonical repository should be hosted - I know some > people have strong opinions that this should be on Zope Foundation > controlled hardware. If that's to be the case then we will need the > svn.zope.org maintainers to setup a shared git repository on that > machine. I think mirroring to github (and/or launchpad in future) will > be the simplest way to provide an anonymously accessible and web > browsable copy. +1 - GitHub is really useful. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 roadmap
On 17 November 2011 14:46, Laurence Rowe wrote: > Here's my current understanding of the Zope 4 roadmap. > > > Zope 4 > -- > > Significant progress has already been made on the following features > and I expect they should all land in time for a Zope 4 release: > > - Storing parent pointers (elro, davisagli): we have a branch of Zope > that changes OFS to store parent pointers on objects implementing > ILocation and fixed the issues that arose in copy/cut/paste and zexp > export code. There's an issue remaining with Acquisition, where we > lost some protection against cycle protection - Hanno continues > working on this. See also: > http://wiki.zope.org/zope2/SetParentAndNameInOFS +1 > - Remove XML zexp export (davisagli). +1 > - Catalog using intids (rossp): we have branches of Zope, ZCatalog and > CMF which change the catalog to use intids as their internal uid and > rid. This needs more testing, but looks very promising. Currently it > uses plone.app.intid / five.intid to maintain an intid to physical > path mapping. Once the parent pointer work is stable, we can stop > doing this and load objects directly from the database connection +0 - can we articulate the benefit of doing this? > - Chameleon (chaoflow): Florian worked on making Zope use Chameleon by > default. This works for the most parts, but there's some problems left > in tests, as Chameleon wants an utility to be registered but a lot of > tests in Zope and CMF don't use ZCML test layers. +1 > - WebOb (garbas): Rok worked on bringing in WebOb as an underpinning > for the request/response objects and making both API's available at > the same time. I think the request is done and a good deal of the > response works. Doing this means we can deprecate parts of the old > request API and encourage the use of the more standard WebOb API. +1, though we'll need to balance the desire to be a better WSGI citizen with adding yet more complexity to BaseRequest and BaseResponse. > - WSGI (matthewwilkes, hannosch): We fixed the bugs that had been > reported about using the current WSGI support (mod_wsgi problems, > forced dependency on repoze.who) on trunk and the 2.13 branch. > Afterwards Matthew continued on a branch to remove the ZServer/medusa > from Zope and leave only the WSGI publisher in place. +1 - I think WSGI should be the only way to deploy Zope 4. > - Decorators for AccessControl (chaoflow). +1 > Future releases (Zope 5, 6, etc.) > - > > There has been some discussion on these topics but not much in the way > of code yet: > > - Traversal Simplification. Remove attribute lookup from default traversal. +1, although this will require a lot of fixes in Zope consumers. I think we need a substantially simpler traversal mechanism that people actually understand. I've pdb'd BaseRequest.traverse more times than I care to remember and I still only vaguely understand it. > - Unicode IDs in OFS +1 > - Remove __allow_access_to_unprotected_subobjects__=1 from OFS.SimpleItem.Item +1, though again will break quite a lot. It's scary from a security perspective, though. > - New ZMI +0 - we certainly need better XSRF protection and accessibility and look and feel are not great, though I think we should also consider what we actually use the ZMI for. A low-level object browser is really useful as a last resort admin tool. A number of the other things (like the various CMF tools, PAS, etc) could just as well expose their own views more independently of the ZMI, though we'd still need a way to discover those views. Perhaps something more simliar to the Plone control panel where tools can register views that just appear in a list of things you may need to manage may be useful, but it'll need some way of rooting that to e.g. Plone sites. > - Move authentication out to WSGI middleware. +1 - anything we can do to make AccessControl simpler and more debuggable would be a big win. > Long term > - > > - Move to WebOb as our request object. The wider Python web framework > community seems to have settled on WebOb as their request object of > choice, so to facilitate sharing of software components with other > projects we should consider making WebOb our request object too > (instead of just wrapping it). This will require buy-in from the wider > ZTK community as the ZTK components will need +1 > - Move to Python 3. +0 Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 release management
On 17 November 2011 15:50, Jim Fulton wrote: > On Thu, Nov 17, 2011 at 7:25 AM, Laurence Rowe wrote: > ... (Interesting roadmap snipped) > >> This process will necessitate a lot of merging, so I want to propose >> that we move to Git for development (something we found very helpful >> at our recent San Francisco Zope 4 sprint.) I don't have any opinion >> on where the canonical repository should be hosted - I know some >> people have strong opinions that this should be on Zope Foundation >> controlled hardware. If that's to be the case then we will need the >> svn.zope.org maintainers to setup a shared git repository on that >> machine. > > Why on that machine? Why not have the ZF set up git.zope.org? > > As the primary maintainer of svn.zope.org, I'm not volunteering > to have more stuff there. :) > >> I think mirroring to github (and/or launchpad in future) will >> be the simplest way to provide an anonymously accessible and web >> browsable copy. > > I haven't used GitHub myself, but I gather it's good. :) Why not > just let them host the project? That's the conclusion Plone came to. We have experience of running such a migration now, so can probably help. We also have some mirroring for backup happening. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 4 release management
On 17 November 2011 16:32, Tres Seaver wrote: >> * Zope 4 will not seek to innovate in itself but encourage innovation >> in software components shared with the wider Python web community. > > > I smell something funny in here: if we aren't innovating, why are we > making the effort? The innovation is in making it possible for users of Zope 2 to better be part of the wide Python web framework community and use tools and frameworks that are also in use in frameworks like Pylons/Pyramid, Django and TurboGears. The other innovation is in making our platform leaner, more maintainable and easier to understand and debug. >> * Zope 4 will not move so quickly that it becomes impossible to make >> Plone, its main consumer, work with it. > > > We should be working to enable the other Zope2-consuming projects to keep > up, too. +1 >> * But neither will Zope 4 seek to retain backwards compatibility at >> any cost. As the basis of Plone 4, Zope 2.13 will see maintenance >> releases as long as Plone 4 is supported. > > > As long as any significant body of developers commits to maintaining it, > even if the Plone devs don't care any more. +1 >> * Zope 4 will not have a release cycle independent of Plone. Zope 4 >> only exists as a transitional path for Zope 2 based applications and >> experience has shown that Zope 2 releases not used in any Plone >> release do not receive the same level of ongoing maintenance. > > > I would actually argue that "Zope4" have no real release cycle at all: > instead, the individual pieces which make up Zope should have their own > cycles, with perhaps a ZTK-like tracking set that Plone and others use as > platform targets. -1 - we'll need something to amalgamate this into a release and we'll need a way to manage and number those releases. >> We want to encourage all features to be developed on separate feature >> branches so we can maintain a stable trunk. Before these feature >> branches are merged they should be posted to the mailing list for >> review. >> >> This process will necessitate a lot of merging, so I want to propose >> that we move to Git for development (something we found very helpful >> at our recent San Francisco Zope 4 sprint.) > > > Note that this question is *not* suitable for "loudest voice on zope-dev > wins" ressolution. The software belongs to the Zope Foundation, which > will make any such decision. The work done on github by the Zope4 > sprinters in SF should be seen as scratchpads for work which will be > migrated back into the canonical repository for each project. > > A couple of points for consideration: > > - - bzr and hg are pretty much isomorphic to git WRT this kind of practice. > Both have claims on our community which Git does not (hg because it is > the tool of choice for Python, bzr because we already use Launchpad). > Note that I use Git daily, and the others somewhat less frequently: I'm > not speaking from ignorance here. > > - - Merging feature branches in SVN is not *that* difficult, typically: I've > done scores of such merges myself with almost no pain (and the really > painful ones would have been pretty much as bad with git / bzr / hg). In the Plone community, we held a poll. GitHub won hands-down. The second choice was staying with self-hosted SVN GitHub is a big leap forward in software project support. It's also rapidly becoming a de-facto place for many people to look for software. We win if the people we want to encourage to fix bugs or contribute features have a low barrier to entry. Note that this also includes Plone developers working on Plone at this stage, since Plone has now moved to GitHub. So, my personal vote would be for Zope to use GitHub (with a backup mirror), allowing me to have a more integrated toolchain. Personally, I find GitHub substantially better than BitBucket, especially for collaboration, and Launchpad nearly unusable. I'd encourage you to look at usage and growth statistics for the three main hosting/collaboration services. >> I don't have any opinion on where the canonical repository should be >> hosted - I know some people have strong opinions that this should be >> on Zope Foundation controlled hardware. > > > I would note that hosting Git repositories without Github reduces the > value proposition substantially: Git's wins in merging are much less > significant than the collaboration workflow changes which github makes > possible (tracking pull requests, in particular). Launchpad provides a > similar mechanism, albeit one which is less sexy to use. OTOH, github's > bug tracker is nothing to write home about, compared to Launchpad. Right - Plone has chosen to stick with self-hosted Trac for bug tracking. I'd imagine Lanchpad remaining Zope's bug tracker. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mai
[Zope-dev] PAS and AccessControl bug?
Hi, I found this code in PAS, which is mostly lifted from AccessControl.userfolder: def _getObjectContext( self, v, request ): """ request -> ( a, c, n, v ) o 'a 'is the object the object was accessed through o 'c 'is the physical container of the object o 'n 'is the name used to access the object o 'v' is the object (value) we're validating access to o XXX: Lifted from AccessControl.User.BasicUserFolder._getobcontext """ if len( request.steps ) == 0: # someone deleted root index_html request[ 'RESPONSE' ].notFoundError( 'no default view (root default view was probably deleted)' ) root = request[ 'PARENTS' ][ -1 ] request_container = aq_parent( root ) n = request.steps[ -1 ] # default to accessed and container as v.aq_parent a = c = request[ 'PARENTS' ][ 0 ] # try to find actual container inner = aq_inner( v ) innerparent = aq_parent( inner ) if innerparent is not None: # this is not a method, we needn't treat it specially c = innerparent elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = aq_inner( v ) # if pub's aq_parent or container is the request container, it # means pub was accessed from the root if a is request_container: a = root if c is request_container: c = root return a, c, n, v Look at this bit again: elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = aq_inner( v ) In AccessControl, it's similar: elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = getattr(v, 'aq_inner', v) Surely, this isn't right? What is the correct thing to do here? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZopeSecurityPolicy global manipulation
Hi, AccessControl.ZopeSecurityPolicy contains this code: from types import MethodType # AccessControl.Implementation inserts: # ZopeSecurityPolicy, getRoles, rolesForPermissionOn from AccessControl.SimpleObjectPolicies import _noroles rolesForPermissionOn = None # XXX: avoid import loop tuple_or_list = tuple, list def getRoles(container, name, value, default): global rolesForPermissionOn # XXX: avoid import loop if rolesForPermissionOn is None: from PermissionRole import rolesForPermissionOn roles = getattr(value, '__roles__', _noroles) if roles is _noroles: if not name or not isinstance(name, basestring): return default if type(value) is MethodType: container = value.im_self cls = getattr(container, '__class__', None) if cls is None: return default roles = getattr(cls, name+'__roles__', _noroles) if roles is _noroles: return default value = container if roles is None or isinstance(roles, tuple_or_list): return roles rolesForPermissionOn = getattr(roles, 'rolesForPermissionOn', None) if rolesForPermissionOn is not None: roles = rolesForPermissionOn(value) return roles Look carefully at how ``rolesForPermissionOn`` is used both at the top, to lazily set a global to avoid an import loop, and the bottom, as an attribute of the ``roles`` object. I'm pretty sure this is wrong™ on many levels, but most importantly, it seems the global is being overwritten each time execution gets down to that last block. I know this module gets munged by Implementation, but I'm pretty sure ImplPython doesn't define getRoles() at least, and I'm not even sure the C implementation does either. To prove it to myself, I made a frivolous equivalent that used 'datetime.date' as the importable. It's a bit ugly, but you get the idea: >>> date = None >>> class C(object): ... def __init__(self, d): ... self.date = d ... >>> c1 = C(lambda: 'x') >>> c2 = C(lambda: 'y') >>> def get(c): ... global date ... if date is None: ... from datetime import date ... date = getattr(c, 'date', None) ... if date is not None: ... print date() ... >>> date is None True >>> get(c1) x >>> date at 0x10dac8140> >>> get(c2) y >>> date at 0x10dac8cf8> >>> Surely, this is all evil volatile? Maybe the global bit just needs to go away? It doesn't seem to be used in that function, and I'm pretty sure the implementation ends up overwriting the global anyway. Cheers, Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope internals documentation
Hi folks, I have documented some of the darker corners of Zope's internals. I put it in the Plone developer documentation for lack of a better place, but it's not Plone-specific: http://collective-docs.readthedocs.org/en/latest/zope_secrets/index.html Topics covered include startup, publication, traversal and security. One reason to do this, apart from morbid fascination, is to provide a baseline against which we can consider simplifying some of this stuff. For example, I'd like to consider an (opt-in) simplification of the publisher and traversal, probably based on a stripped-down and modernised repoze.zope2, which does away with some hooks and edge cases, but is much simpler and easier to understand. Some things we could consider chopping are: - Attribute traversal to anything other than methods at the end of the traversal chain (i.e. use __getitem__ traversal only) - Traversal to anything without explicit security declarations - The docstring security check - Maybe __bobo_traverse__ (i.e. just implement __getitem__) and __before_publishing_traverse__ (use a BeforeTraverseEvent instead, and notify this for all traversals, not just over local component sites) - All differences between publication and path traversal This is still somewhat half-baked and obviously would break things and require at least a new major version of Zope, but I think it's worth exploring at least. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 09:44, Jens Vagelpohl wrote: > > On Dec 31, 2011, at 20:09 , Martin Aspeli wrote: > >> Hi folks, >> >> I have documented some of the darker corners of Zope's internals. I >> put it in the Plone developer documentation for lack of a better >> place, but it's not Plone-specific: >> >> http://collective-docs.readthedocs.org/en/latest/zope_secrets/index.html > > Hi Martin, > > There *is* a better place, docs.zope.org. If you can tell me where the > sources are I can put it there. Sure: Clone https://github.com/collective/collective.developermanual/ and get it from source/zope_secrets. Where is docs.zope.org maintained? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 10:43, Jens Vagelpohl wrote: > Hi Martin, > >>> There *is* a better place, docs.zope.org. If you can tell me where the >>> sources are I can put it there. >> >> Sure: Clone https://github.com/collective/collective.developermanual/ >> and get it from source/zope_secrets. > > Thanks, I'll take a look at it today. > > >> Where is docs.zope.org maintained? > > On one of the ZF servers. If it's going to go there, I'd like it to (a) be in version control and (b) be somewhere that I can edit it. Is that doable? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 10:51, Andreas Jung wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > > Martin Aspeli wrote: >> On 1 January 2012 10:43, Jens Vagelpohl wrote: >>> Hi Martin, >>> >>>>> There *is* a better place, docs.zope.org. If you can tell me >>>>> where the sources are I can put it there. >>>> Sure: Clone >>>> https://github.com/collective/collective.developermanual/ and get >>>> it from source/zope_secrets. >>> Thanks, I'll take a look at it today. >>> >>> >>>> Where is docs.zope.org maintained? >>> On one of the ZF servers. >> >> If it's going to go there, I'd like it to (a) be in version control >> and (b) be somewhere that I can edit it. Is that doable? > > Wouldn't it make sense to integrate your docs with The Zope Book. > It's maintained using Sphinx and the sources are on svn.zope.org > (somewhere). This is really low level documentation. The Zope Book is for people using Zope. This is for people who may need to maintain or deep-debug it. I'm happy for it to be integrated if people think it makes sense, but I think it may be quite off-putting to read what is in many places block-by-block explanations of what the code does. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 1 January 2012 11:00, Jens Vagelpohl wrote: > > On Jan 1, 2012, at 11:46 , Martin Aspeli wrote: >>>> Where is docs.zope.org maintained? >>> >>> On one of the ZF servers. >> >> If it's going to go there, I'd like it to (a) be in version control >> and (b) be somewhere that I can edit it. Is that doable? > > That's how we do it with almost everything underneath the docs.zope.org > hostname. The sources are on svn.zope.org are are pulled/built regularly. > Where the source comes from doesn't really matter. The only requirement is > that it should be a scriptable buildout process, like a buildout/Sphinx setup. Sphinx on svn.zope.org works for me. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope 2 WSGI investigation
Hi, There are three known WSGI implementations of the Zope 2 publisher. I've had a look at them and made some notes about what I think provides the best story: ## Zope 2.13 WSGIPublisher Pros: * Allows distributed transaction management with repoze.tm2 * Allows distributed retry with repoze.retry * Ships with Zope * Quite simple Cons: * Requires repoze.tm2 and repoze.rety * Does not properly emit publication events - possible patch in https://gist.github.com/1548061 * Does not do error handling or exception views * Claims not to properly implement streaming (though there is some code for it) * Probably less well tested than infrae.wsgi and repoze.zope2 (at least there is zero documentation) ## infrae.wsgi Pros: * Clean and well documented * Properly emits publication events * Supports streaming * Supports simplified virtual hosting with X-VHM-Host * Supports exception handling / error views * Reportedly has significant production use Cons: * Not 100% compatible (but close and fixable) - fix to make plone.transformchain work is here: https://gist.github.com/1547328 * Unnecessary five.grok dependency (but easy to rewrite to use ZCML registration) * No support for middleware transaction and retry management, so these can't be distributed across a WSGI pipeline * Error logging will not support ZMI error_log and assumes single process * Error handling is slightly different to standard publisher's exception views, and also does not honour existing standard_error_message etc ## repoze.zope2 Pros: * Clean and well documented * Reimplements and simplifies the BaseRequest.traverse() code, with comments * Supports distributed transaction management and retry Cons: * Replicates a lot of Zope startup code * Has now-unnecessary code to manage instances and configuration * repoze.obob abstraction is unnecessary since nothing else uses this * Does not emit publication events - possible patch in http://bugs.repoze.org/issue181 * Does not do error handling or exception views * Problems with file resources (does not properly traverse to browserDefault() result) -- possible patch in http://bugs.repoze.org/issue64 * Requires various middleware (repoze.tm, repoze.retry, repoze.vhm) ## Suggested approach going forward * Integrate infrae.wsgi into Zope 2 * Remove its five.grok dependency * Use the same exception-views protocol as ZPublisher (mainly, that the view name is ``index.html``) * Stop using __ 'private' variables in response.py to make it easier to work with * Add some BBB support for existing error logging and error messages Thoughts? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2 WSGI investigation
On 2 January 2012 08:50, Wichert Akkerman wrote: > On 01/01/2012 08:39 PM, Martin Aspeli wrote: >> >> Hi, >> >> There are three known WSGI implementations of the Zope 2 publisher. >> I've had a look at them and made some notes about what I think >> provides the best story: >> >> ## Zope 2.13 WSGIPublisher >> >> Pros: >> >> * Allows distributed transaction management with repoze.tm2 >> * Allows distributed retry with repoze.retry >> * Ships with Zope >> * Quite simple >> >> Cons: >> >> * Requires repoze.tm2 and repoze.rety > > > Why is that a con? I use repoze.tm2 and repoze.retry with all my pyramid > projects and they work beautifully. Only insofar as it means you have to have these in your WSGI pipeline or it won't work, so there are more places things can go wrong. You'll note that I also consider not supporting such things a con for infrae.wsgi. I wouldn't get too hung up on it, I was mainly just trying to bring out the differences. It'd be nice if it wasn't a hard requirement, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope internals documentation
On 2 January 2012 12:33, Jens Vagelpohl wrote: > Hi Martin, > >> Sphinx on svn.zope.org works for me. :) > > I have created a simple buildout and put it in SVN: > > http://svn.zope.org/zope_secrets/ > > The output is shown at http://docs.zope.org/zope_secrets/ and linked from the > front page at http://docs.zope.org/. Every 6 hours, a cron job looks to see > if the SVN revision has changed and if it has then the output is regenerated. Thanks! Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2 WSGI investigation
On 3 January 2012 06:39, Chris McDonough wrote: > On Mon, 2012-01-02 at 10:39 +0000, Martin Aspeli wrote: >> On 2 January 2012 08:50, Wichert Akkerman wrote: >> > On 01/01/2012 08:39 PM, Martin Aspeli wrote: >> >> >> >> Hi, >> >> >> >> There are three known WSGI implementations of the Zope 2 publisher. >> >> I've had a look at them and made some notes about what I think >> >> provides the best story: >> >> >> >> ## Zope 2.13 WSGIPublisher >> >> >> >> Pros: >> >> >> >> * Allows distributed transaction management with repoze.tm2 >> >> * Allows distributed retry with repoze.retry >> >> * Ships with Zope >> >> * Quite simple >> >> >> >> Cons: >> >> >> >> * Requires repoze.tm2 and repoze.rety >> > >> > >> > Why is that a con? I use repoze.tm2 and repoze.retry with all my pyramid >> > projects and they work beautifully. >> >> Only insofar as it means you have to have these in your WSGI pipeline >> or it won't work, so there are more places things can go wrong. >> >> You'll note that I also consider not supporting such things a con for >> infrae.wsgi. I wouldn't get too hung up on it, I was mainly just >> trying to bring out the differences. It'd be nice if it wasn't a hard >> requirement, though. > > FWIW, when I wrote repoze.tm2 I did not know that the transaction module > already supported retry, so at least repoze.rety should die in favor of > logic in something-like-repoze.tm2 that looks a little more like this: > > https://github.com/Pylons/pyramid_tm/blob/master/pyramid_tm/__init__.py#L49 > > (that's obviously not WSGI middleware, I'm just showing what the general > logic should look like) Am I right in thinking Pyramid no longer uses repoze.tm2 or a middleware approach? What was the rationale for that design decision? Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2 WSGI investigation
On 3 January 2012 08:01, Chris McDonough wrote: >> Am I right in thinking Pyramid no longer uses repoze.tm2 or a >> middleware approach? What was the rationale for that design decision? > > You're right, Pyramid scaffolding no longer supplies repoze.tm2 or any > other WSGI middleware component for handling transaction commits/aborts. > Instead, it uses a Pyramid "tween", which is sort of like internal > Pyramid middleware inasmuch as its a user-manageable functional > composition terminating at the application. See > http://www.slideshare.net/aconrad/alex-conrad-pyramid-tweens-ploneconf-2011 > > Tweens can be reordeded arbitrarily, and the Pyramid exception handling > logic (which locates and executes a Pyramid view when an application > exception is raised) is itself a tween. > > The rationale is that application-specific error logic often needs to be > executed after the transaction has been committed or aborted due to an > exception. For example, if an exception is raised by an application, > you'd like the system to abort the transaction, then potentially display > a custom internal server error page (e.g. twitter fail-whale). People > want to be able to write the error logic within their existing Pyramid > development environment (where they have access to templating systems, a > real request object, and possibly their database and other niceties) > instead of as WSGI middleware higher in the stack than the transaction > manager. I mostly agree with this, and for this reason I think the ZPublisher.Pubevents (not quite as good as tweens, I admit) and exception views are useful. With a middleware solution, you may find yourself passing application state in the wsgi environ which seems wrong. > FWIW, I believe Tres still disagrees strongly with this decision. I can certainly see the use case for distributing transactions across middleware and/or composite apps, which is why it may make sense for this to be opt-in. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
