Re: [Zope-dev] zope-tests - FAILED: 4, OK: 8
On Tue, Sep 10, 2013 at 09:32:18AM +0300, Marius Gedminas wrote:
On Mon, Sep 09, 2013 at 12:26:33PM -0400, Tres Seaver wrote:
On 09/09/2013 11:53 AM, Marius Gedminas wrote:
On Sat, Sep 07, 2013 at 08:27:04PM -0400, Tres Seaver wrote:
On 09/07/2013 07:00 PM, Zope tests summarizer wrote:
[1]FAILED winbot / zope.app.authentication_py_265_32
https://mail.zope.org/pipermail/zope-tests/2013-September/078621.html
[2]FAILED winbot / zope.app.http_py_265_32
https://mail.zope.org/pipermail/zope-tests/2013-September/078622.html
[3]FAILED winbot / zope.app.publication_py_265_32
https://mail.zope.org/pipermail/zope-tests/2013-September/078620.html
[4]FAILED winbot / zope.app.wsgi_py_265_32
https://mail.zope.org/pipermail/zope-tests/2013-September/078623.html
All four show the same failure in the bulidout phase:
% ---
While: Installing test. Error: There is a version conflict. We
already have: webtest 2.0.7
% ---
Could be my fault. I released zope.testbrowser 4.0.3 to PyPI a few
days ago, and it has install_requires = ['WebTest = 1.3.4'].
OTOH it wasn't me who added that = constraint, that happened a year
ago:
https://github.com/zopefoundation/zope.testbrowser/commit/699a18bb2868d23cd7ab3da80cf394b75f977d91
I just made a release with all the accumulated changes.
Looks like that was Adam: maybe we should be trying to push out a
version of z.tb compatible with current WebTest?
IIRC there's a brand new zope.testbrowser 4.1.0 with Python 3 support,
built on top of WebTest, waiting in the py3 branch for... Actually, I'm
not sure what it is waiting for. (I thought ZODB with Python 3 support,
but it doesn't look like zope.testbrowser depends on ZODB, at least not
directly.)
Indeed, and it's had no changes for at least 5 months and been tested
with a few existing codebases.
+1 for releasing it
Andrey?
Marius Gedminas
--
http://pov.lt/ -- Zope 3/BlueBream consulting and development
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Switching zope.testbrowser to webtest instead of mechanize
On Wed, Apr 17, 2013 at 11:15:35PM +0300, Andrey Lebedev wrote: On 04/15/2013 01:42 PM, Brian Sutherland wrote: Great! Now, as all our webtest fixes are released upstream, I plan to make an alpha release of what we've done so far. If you have any outstanding issues with new testbrowser, please report. Just committed the last two. Sorry for the drip feed, but it takes a while to make simple test cases from the failures I see in my app. Just fixed those. Great, I've just confirmed that I have no more test failures in my app caused by bugs in the new testbrowser. I've found two that are bugs in the old testbrowser though, where tests passed by mistake ;) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Switching zope.testbrowser to webtest instead of mechanize
On Sat, Apr 13, 2013 at 12:54:22AM +0300, Andrey Lebedev wrote: On 04/12/2013 02:59 PM, Brian Sutherland wrote: This is looking very very good, in my app very few tests are still failing. Great! Now, as all our webtest fixes are released upstream, I plan to make an alpha release of what we've done so far. If you have any outstanding issues with new testbrowser, please report. Just committed the last two. Sorry for the drip feed, but it takes a while to make simple test cases from the failures I see in my app. +1 to an alpha release soon. I poked around at one and found that the way non-ascii in forms is handled has changed under python 2, here's a test that shows the problem: https://github.com/zopefoundation/zope.testbrowser/commit/e0e965086c7fded415963518a6adef41b9d14619 Thanks, I've fixed the testbrowser according to your testcase. perfect I'm not sure what the intention of the python 3 port is wrt unicode. Have you had any thoughts on the matter? The idea is to make testbrowser to return/accept native strings instead of bytes/unicode for both py2 and py3. Under py2, controls will return str objects (for backwards compatibility), and under py3 that would be unicode strings (for usage convenience). I guessed correctly then :) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Switching zope.testbrowser to webtest instead of mechanize
On Tue, Mar 19, 2013 at 04:40:02PM +0200, Andrey Lebedev wrote: On 03/04/2013 08:40 PM, Andrey Lebedev wrote: As you may already know, there is an effort to port zope packages to Python 3 going on. As part of this effort we want to port zope.testbrowser. Status update: test suite passes on webtest backend under python-2.6, 2.7 and 3.3. Further things to do: make some performance improvements, try some real-world tests and release a alpha version. This is looking very very good, in my app very few tests are still failing. I poked around at one and found that the way non-ascii in forms is handled has changed under python 2, here's a test that shows the problem: https://github.com/zopefoundation/zope.testbrowser/commit/e0e965086c7fded415963518a6adef41b9d14619 I'm not sure what the intention of the python 3 port is wrt unicode. Have you had any thoughts on the matter? -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Switching zope.testbrowser to webtest instead of mechanize
On Tue, Mar 19, 2013 at 04:40:02PM +0200, Andrey Lebedev wrote: On 03/04/2013 08:40 PM, Andrey Lebedev wrote: As you may already know, there is an effort to port zope packages to Python 3 going on. As part of this effort we want to port zope.testbrowser. Status update: test suite passes on webtest backend under python-2.6, 2.7 and 3.3. Further things to do: make some performance improvements, try some real-world tests and release a alpha version. I tried it against my app with good results so far :) I did notice two regression and pushed failing tests to you py3 branch. It seemed the easiest/fastest way forward, if you'd rather I made a separate branch for these, let me know. -- Andrey Lebedev aka -.- . -.. -.. . .-. Software engineer Homepage: http://lebedev.lt/ Jabber ID: ked...@jabber.ru ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Switching zope.testbrowser to webtest instead of mechanize
On Mon, Mar 04, 2013 at 08:40:23PM +0200, Andrey Lebedev wrote: Hello zope-dev, As you may already know, there is an effort to port zope packages to Python 3 going on. As part of this effort we want to port zope.testbrowser. Great! I'm very interested in this, I've also got an app which I'll need to port to python3 at some time. Unfortunately, mechanize package, that zope.testbrowser depends on has no py3 support. There is an unfinished effort at [1], however, after closer examination of the codebase and state of the port, it seems more practical to switch internal implementation of zope.testbrowser to webtest. So far, it seems webtest has all the features needed to do the switch. I started a branch at github ([2]) to track work in progress. The goal is to make most (say 80%) of existing tests to work without modifications under new implementation. Looking at the branch, it seems that this is basically going to be a re-write of the existing code. You're planning to keep the mechanize backend available? Might I suggest zope.testbrowser.webtest for the WebTest backed testbrowser rather than zope.testbrowser.browser2? Could you also keep the tests for the mechanize testbrowser intact? They could be skipped if mechanize is not importable and mechanize downgraded to an optional dependency. [1] http://web.cecs.pdx.edu/~adevore/mechanize/ [2] https://github.com/zopefoundation/zope.testbrowser/tree/webtest -- Andrey Lebedev aka -.- . -.. -.. . .-. Software engineer Homepage: http://lebedev.lt/ Jabber ID: ked...@jabber.ru ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Switching zope.testbrowser to webtest instead of mechanize
On Tue, Mar 05, 2013 at 05:56:54PM +0200, Andrey Lebedev wrote: On 03/05/2013 05:10 PM, Brian Sutherland wrote: So far, it seems webtest has all the features needed to do the switch. I started a branch at github ([2]) to track work in progress. The goal is to make most (say 80%) of existing tests to work without modifications under new implementation. Looking at the branch, it seems that this is basically going to be a re-write of the existing code. Yes, it is a re-write, but not re-write from scratch. I use existing code as much as possible. You're planning to keep the mechanize backend available? Initially we didn't plan to keep mechanize. I think it would add an unjustified burden of choice to users (which backend to pick, if they work the same way?) and mechanize backend will/may not work in python3 anyway. The question of how compatible the webtest-based implementation will be, is still an open one though. However, we have a big real-world project to test compatibility against. You're right, the big question is how compatible the WebTest version will be. I don't think anyone will object to a drop-in replacement. Having to modify 20% of existing tests sounds like a lot. I guess we'll have to see how compatible the new browser will be. I'm willing to pitch in by testing it against my own application, please let me know when/if you want me to do that. Might I suggest zope.testbrowser.webtest for the WebTest backed testbrowser rather than zope.testbrowser.browser2? I admit there is a bit of mess right now in the branch as I need to preserve existing code to copy when applicable. My plan is to rename z.t.browser2 to z.t.browser eventually, so it is not a final name. Ok! Could you also keep the tests for the mechanize testbrowser intact? They could be skipped if mechanize is not importable and mechanize downgraded to an optional dependency. Again, if the compatibility of new implementation is proven to be good enough, I see no reason to keep mechanize around. I might miss some important reasons, though. One thing which I'm not sure you can do with a webtest backend is using it as a real browser. i.e. you'll probably not get over_the_wire.txt to pass. However, I don't know of anyone who actually uses that functionality. Hmm, though I guess you can have a WSGI application that fires off real HTTP requests... -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.security
On 02/11/2013 03:36 PM, Stephan Richter wrote: Hi Tres (and everyone else reading along), as you are probably aware, zope.security is a package that blocks a lot of other packages from being ported. I just checked out the coverage on Github and it looks like you are making good progress. The biggest issue I see with zope.security is its dependency on RestrictedPython, because I think that will take a long time to port correctly. However, most of zope.security is very much usable and used without zope.security.untrustedpython. I propose to split zope.security.untrustedpython into a separate package called zope.untrustedpython, so that a port of zope.security to Python 3 can move forward. (Note: I am signing up for the work.) +1 ! -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] How to signal that projects have moved to github
On Thu, Jan 10, 2013 at 09:15:21AM -0500, Jim Fulton wrote: I'd like us to agree on how we're going to indicate in SVN that projects have been moved to github. Here's an opening bid: - Replace contents of trunk with a single file MOVED_TO_GITHUB containing the URL if the project page in github - Copy above file to root of project. - Make project read only +1 An argument against cleaning out trunk will break CI tools. This is also an argument *for* cleaning out trunk. :) Maybe a bit draconian at this stage? This could perhaps be done much later in the migration by scripting the removal of all trunks with a MOVED_TO_GITHUB file. Thoughts? There are a lot of other details of the migration that need to be worked out, like how folks should request migration and automating the conversion further. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.interface 4.0.0 considered annoying.
On Sun, May 20, 2012 at 12:08:14AM -0400, Tres Seaver wrote: On 05/19/2012 07:23 PM, Jim Fulton wrote: On Sat, May 19, 2012 at 5:59 PM, Tres Seaver tsea...@palladion.com wrote: On 05/19/2012 12:12 PM, Jim Fulton wrote: The declaration-syntax warnings in the zope.interface 4.0.0 are annoying and mostly pointless. There will be enough pain in porting applications to Python 3, avoiding the surprise of having to use a different interface-declaration syntax will not be of any consequence. While the deprecation warning may provide some tiny benefit, the annoyance, especially for applications or libraries that still support Python 2.5 far outweights the benefit. We're in an odd spot: the Python3 support for zope.interface mostly works, but the class advice bits (implements, implementsOnly, classProvides) fail silently under Python3 before 4.0.0: they literally have no effect due to changes in class initialization machinery. zope.interface 4.0.0 changes those silent failures into exceptions (which is what I thought you were going to complain about). I'd forgotten the fail silently bit. Can't we make the fail loudly for Python 3? They already do: 4.0.0 makes them exceptions under Python3: that fact was actually my rationale for the warnings under Python2, to allow folks to find and remediate them before porting. I will release a 4.0.1 silencing the warnings unless somebody speaks up in their favor. +1 to silencing the warnings under Python 2 -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Python 3.3 port of zope.configuration
On Fri, May 04, 2012 at 07:50:57PM -0400, Tres Seaver wrote:
On 04/26/2012 02:34 PM, Tres Seaver wrote:
On 04/26/2012 01:50 PM, Brian Sutherland wrote:
On Thu, Apr 26, 2012 at 12:06:13PM -0400, Tres Seaver wrote: On
04/26/2012 04:39 AM, Brian Sutherland wrote:
On Thu, Apr 12, 2012 at 07:37:46PM -0400, Tres Seaver wrote:
Also, do your branches still pass tests on 3.2.x? I'm sort
of keen to see both zope.schema and zope.configuration join
the 2to3-free camp with zope.event, zope.interface,
zope.i18nmessageid, and zope.exceptions (yes,
zope.configuration, I'm looking at you next).
I see you've started doing this in a better and more rigorous
way than I was. So I'm going to abandon my plans to merge in
favour of your branches.
I'd still urge you to have a look at my zope.schema branch,
which I think solves a real problem with URI fields and
Python 3. The patch is pretty small.
My zope.schema branch is largely focused on improving the unittest
coverage, while also making the package work with Python 3.2 (i.e.,
not relying on 3.3's restoration of unicode literals).
I would be glad to look at your branch; I had already considered
making the native string fields APIs. I'm not sure that I agree
that the ASCII / URI / Id / DottedName fields should have native
string as its type: can you help by elaborating there?
For DottedName we're basically forced to use native strings as
you can import modules with unicode names on python 3:
from 漢語 import Español print(Español)
class '漢語.Español'
Id is either a URI or a DottedName. So it must, like DottedName,
also be a native string.
As for URI, python 3 library functions seem to not care, so we can
choose either:
from urllib.parse import urlparse
urlparse('http://www.example.com/unicode')
ParseResult(scheme='http', netloc='www.example.com',
path='/unicode', params='', query='', fragment='')
urlparse(b'http://www.example.com/bytes')
ParseResultBytes(scheme=b'http', netloc=b'www.example.com',
path=b'/bytes', params=b'', query=b'', fragment=b'')
But, for consistency's sake, URI, Id and DottedName should be the
same. That makes the best choice for URI a native string.
Another more practical argument is that having URI be a native
string made the porting of zope.configuration to python 3 much
easier. There were a few nasty bugs when trying it with URI as
bytes.
I have no strong argument for ASCII.
FWIW, here's the original discussion on porting zope.schema:
https://mail.zope.org/pipermail/zope-dev/2011-October/043540.html
Thanks very much for the explanation. I will plan to merge in your
changes in my branch ASAP.
Done and merged to the trunk with my 'test_cleanup' branch.
Looks great, thanks!
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Python 3.3 port of zope.configuration
On Thu, Apr 12, 2012 at 07:37:46PM -0400, Tres Seaver wrote: Also, do your branches still pass tests on 3.2.x? I'm sort of keen to see both zope.schema and zope.configuration join the 2to3-free camp with zope.event, zope.interface, zope.i18nmessageid, and zope.exceptions (yes, zope.configuration, I'm looking at you next). I see you've started doing this in a better and more rigorous way than I was. So I'm going to abandon my plans to merge in favour of your branches. I'd still urge you to have a look at my zope.schema branch, which I think solves a real problem with URI fields and Python 3. The patch is pretty small. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Python 3.3 port of zope.configuration
On Thu, Apr 26, 2012 at 12:06:13PM -0400, Tres Seaver wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 04/26/2012 04:39 AM, Brian Sutherland wrote:
On Thu, Apr 12, 2012 at 07:37:46PM -0400, Tres Seaver wrote:
Also, do your branches still pass tests on 3.2.x? I'm sort of keen
to see both zope.schema and zope.configuration join the 2to3-free
camp with zope.event, zope.interface, zope.i18nmessageid, and
zope.exceptions (yes, zope.configuration, I'm looking at you next).
I see you've started doing this in a better and more rigorous way than
I was. So I'm going to abandon my plans to merge in favour of your
branches.
I'd still urge you to have a look at my zope.schema branch, which I
think solves a real problem with URI fields and Python 3. The patch
is pretty small.
My zope.schema branch is largely focused on improving the unittest
coverage, while also making the package work with Python 3.2 (i.e., not
relying on 3.3's restoration of unicode literals).
I would be glad to look at your branch; I had already considered making
the native string fields APIs. I'm not sure that I agree that the
ASCII / URI / Id / DottedName fields should have native string as its
type: can you help by elaborating there?
For DottedName we're basically forced to use native strings as you can
import modules with unicode names on python 3:
from 漢語 import Español
print(Español)
class '漢語.Español'
Id is either a URI or a DottedName. So it must, like DottedName, also be
a native string.
As for URI, python 3 library functions seem to not care, so we can
choose either:
from urllib.parse import urlparse
urlparse('http://www.example.com/unicode')
ParseResult(scheme='http', netloc='www.example.com', path='/unicode',
params='', query='', fragment='')
urlparse(b'http://www.example.com/bytes')
ParseResultBytes(scheme=b'http', netloc=b'www.example.com', path=b'/bytes',
params=b'', query=b'', fragment=b'')
But, for consistency's sake, URI, Id and DottedName should be the same.
That makes the best choice for URI a native string.
Another more practical argument is that having URI be a native string
made the porting of zope.configuration to python 3 much easier. There
were a few nasty bugs when trying it with URI as bytes.
I have no strong argument for ASCII.
FWIW, here's the original discussion on porting zope.schema:
https://mail.zope.org/pipermail/zope-dev/2011-October/043540.html
(Also, looks like we should
promote the 'INativeString' interface to an API).
+1
Tres.
- --
===
Tres Seaver +1 540-429-0999 tsea...@palladion.com
Palladion Software Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk+ZcnUACgkQ+gerLs4ltQ4BWgCg0YOBWiGBJt8UIgBYG1F/sqsh
qqoAoI6H77dS3EOOYFnOiQgEP6jddsNB
=0Xwc
-END PGP SIGNATURE-
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Python 3.3 port of zope.configuration
On Thu, Apr 12, 2012 at 07:37:46PM -0400, Tres Seaver wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 04/11/2012 03:57 AM, Brian Sutherland wrote:
Hi,
I'd like to merge 2 branches which get zope.configuration's tests
running on Python 3.3. Here is an overview (with patches attached):
zope.schema:
*
svn+ssh://svn.zope.org/repos/main/zope.schema/branches/jinty-native_string
* This branch corrects a mistake I think I made in the zope.schema
Python3 port. After my experience with the zope.configuration port I
now think that the URI field should be a native string rather than
bytes.
Unfortunately this change is backwards incompatible for Python 3 users
of zope.schema. But I think the number users of zope.schema under
Python 3 are rather small at this point.
* I make public NativeString and NativeStringLine classes. These are
equivalent to the Bytes type under Python 2 and the Text type under
Python 3. Please devote some bikeshedding energy to better names for
these classes ;)
zope.configuration:
*
svn+ssh://svn.zope.org/repos/main/zope.configuration/branches/jinty-python3
* Adds a dependency on six
* Drops Python 2.4 and 2.5 compatibility
To run the tests of the zope.configuration branch you currently need
to use the zope.interface trunk.
If there are no objections, I'll merge in a few weeks or so.
How deep is the six dependency? For every package I've ported to date,
it has ended up being more sensible to implement a local '_compat' module
which had only the needed straddles (sometimes inspired by six, sometimes
very localized).
I'm sure you can get away without it. But having functions from six
copied into huge numbers of zope.* packages made my skin crawl. As Jim
said a huge DRY violation.
Also, do your branches still pass tests on 3.2.x?
Yes for zope.schema
No for zope.configuration, but that can be fixed by changing u'' to
u('') in a lot of places.
I'm sort of keen to
see both zope.schema and zope.configuration join the 2to3-free camp
zope.schema's already there, my branch is only fixing what I consider to
be a bug in the Python 3 port.
with zope.event, zope.interface, zope.i18nmessageid, and zope.exceptions
(yes, zope.configuration, I'm looking at you next).
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding broken/missing support to zope.interface?
On Wed, Apr 11, 2012 at 09:30:36AM -0700, Ross Patterson wrote: Hanno Schlichting ha...@hannosch.eu writes: On Mon, Apr 9, 2012 at 10:33 PM, Tres Seaver tsea...@palladion.com wrote: Persistent component registries are not a good enough reason to add such coupling (I'd be in favor of splitting support for persistent registries out of zope.component, too, while we're at it). Let's put the broken support into code which depends on zope.interface, zope.component, and the ZODB, and invert the dependency by having the new code install something into the base code which provides the desired support: the only change to zope.interface should be documenting the insertion point, and testing that it does the right thing when a dummy is plugged into it. I looked at the possible contenders for that dependency description. The ZODB depends on zope.interface itself, though not on zope.component. zope.container is the one that has the most minimal dependencies, while still relying on zope.component and the ZODB. zope.site depends on zope.container, but given its scope sounds like the better place to me. I vaguely remember us discussing to move persistent registries into zope.site at some point. Since we moved zope.site.hooks into zope.component, zope.site doesn't have much else to do anymore. Apart from those two, there's a whole lot more that have far more dependencies or are unrelated in scope, like zope.annotation or zope.catalog. This problem isn't so much ZODB specific as it is specific to pickling. The problem I don't know how to solve without modifying zope.interface is that the on pickle end of things, the only hook I'm aware of is on the unpickling side, namely overriding `find_global` as ZODB does. But there's no way for `find_global` to know that the given global should be an interface just from the module and name which is what the pickle contains. We need to hook into the process at the time the object is pickled. As far as I can see the only way to do that is through the object's __reduce__ method. As such, the only options I see are to add something conditional to `zope.interface.InterfaceClass.__reduce__` or to make `zope.interface.InterfaceClass.__reduce__` hookable in some way. Would the latter address the concerns people are raising here? Tres was suggesting something like that. It would address my concerns. If so, what's the right way to approach implement that? I think you need someone intimate with the ZODB to suggest that. One way might be something like the adapter_hooks already present in interface.py. That would at least be consistent, but there are probably many good reasons to not do it that way. That code could look something like: reduce_hooks = [] class InterfaceClass: def __reduce__(self): for hook in reduce_hooks: result = hook(self) if result is not None: return result return self.__name__ Just monkey patching from ZODB to zope.interface? Indeed, that is also another option. Ross ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding broken/missing support to zope.interface? (was: experimental.broken - Graceful handling of broken interfaces and components in the ZODB)
On Mon, Apr 09, 2012 at 08:15:16PM +, Martin Aspeli wrote: On 9 April 2012 15:41, Brian Sutherland br...@vanguardistas.net wrote: On Sun, Apr 08, 2012 at 01:04:37PM -0700, Ross Patterson wrote: experimental.broken is working well for me. It greatly aided me in getting through a particularly nasty upgrade allowing me to cleanup the ZCA cruft left by poorly behaved add-ons. I'd like to proceed with adding the core of this to zope.interface and I need the developers/maintainers to weigh in. The first and most fundamental matter is changing interface pickles such that they can be unpickled into something that can fulfill the minimum interface contract and don't break the ZCA. To that end, I'd like to add the following to zope.interface.interface: ... try: from ZODB.broken import find_global from ZODB.broken import IBroken def find_interface(modulename, globalname, Broken=IBroken, type=InterfaceClass): Find a global interface, returning a broken interface if it can't be found. return find_global(modulename, globalname, Broken=IBroken, type=InterfaceClass) except ImportError: IBroken = Interface def find_interface(modulename, globalname, Broken=IBroken, type=InterfaceClass): Find a global interface, raising ImportError if it can't be found. # From pickle.Unpickler.find_class __import__(module) mod = sys.modules[module] klass = getattr(mod, name) return klass ... class InterfaceClass(Element, InterfaceBase, Specification): ... def __reduce__(self): if self is IBroken: return self.__name__ return (find_interface, (modulename, globalname)) ... -1 For a number of reasons, but mainly because you add a test dependency on the ZODB from zope.interface. I think that zope.interface is such a fundamental package and the dependency is unacceptable. It's a soft dependency only, looking at the code sample. Soft dependencies are cheating ;) Cheating is great when you are refactoring old code to try to minimize dependencies. For new code it's better to just do it right. There has lately been a LOT of work done reducing the dependency structure of zope.* packages. You need a VERY good reason to start reversing that. It doesn't add any more (required) dependencies. It is required the moment you actually want to write a test for your code. That means it's pretty much required for development. Also, if you of the more fanatical sort that believe you should run what you test and test what you run then even a testing dependency is a real dependency. This is a real issue that is breaking the sites of a lot of real users of zope.interface in a way that is hard to debug and reverse. It is quite ridiculous that something as fundamental as zope.interface needs to have knowledge of a specific database implementation to avoid problems that are hard to debug and reverse. Can you think of a better way to get around it? Other than don't get into the situation which isn't a valid answer as long as the ZTK ecosystem supports persistent local components. I'm sorry, having never actually used the ZODB I have absolutely no idea on what level this could best be implemented. A question, do you need this code enabled all the time? Or is it something you only need during specific moments like upgrades? -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding broken/missing support to zope.interface? (was: experimental.broken - Graceful handling of broken interfaces and components in the ZODB)
://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Merge proposal: tseaver-better_unittests branch of zope.interface
On Mon, Mar 26, 2012 at 05:38:07PM -0400, Tres Seaver wrote: In addition to minimizing Zope-iness, providing full coverage using small, descriptively-named unittests makes the code more maintainable. For instance, I expect to build on top of these improved tests as the basis for a conversion to a subset, supporting Python 2.6, 2.7, and 3.x from a single codebase, without needing a translator like lib2to3. I think it will also be easier to improve the docs, now that they no longer bear the burden of supplying coverage / regression testing for the code. We can remove a bunch of extremely-terse fragments, and have the examples which remain focus more on improving the reader's understanding than exercising some corner case. Unless the consensus is against it, I plan to merge this branch to the trunk early next week. This sounds great, I think it's exactly the right way to go. It's just a LOT of work, a BIG thanks for taking it on! -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New release for zope.schema
On Thu, Mar 22, 2012 at 03:27:20PM +0100, Jan-Carel Brand wrote: (Pdb) vocabulary.__provides__.__iro__ (InterfaceClass zope.schema.interfaces.IContextSourceBinder, InterfaceClass zope.interface.Interface) There it *does* provide IContextSourceBinder. Any ideas why this vocabulary doesn't provide it in Python 3? A variant of this one, perhaps: https://bugs.launchpad.net/zope.interface/+bug/911851 -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope] mod_rewrite and IIS7
I am attempting to park Zope behind Windows 2008/IIS7 (using its mod_rewrite function) and seem to be having some difficulty getting it to work. What I want to do is make directory on Zope website look like it is a directory on an IIS7 website. I seem to be close but missing some critical element. Does anybody have a step by step process for doing this or samples of rules on IIS7 for doing something similar? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] zope.server still used?
On Tue, Dec 20, 2011 at 08:08:36AM +0100, Wolfgang Schnerring wrote: * Chris McDonough chr...@plope.com [2011-12-18 03:57]: Is zope.server still largely used by e.g. bluebream, grok, and other zope 3 apps? Or do people tend to use other WSGI servers instead? We're using zope.server on several Zope 3 projects (that were begun before grok or WSGI or anythin existed ;), and it's *ahem* serving us just fine, no hiccups, no nothing, just doing its job. (We, too, see the occasional broken pipe error in our logs like Marius mentioned, I think that's due to requests that were aborted client-side, thus more an annoyance than anything.) On the other hand, we still haven't found a WSGI server we're happy with for deployment (as opposed to development). Paster has been known to randomly die on us (yup, I've only got anecdotes and nothing hard and fast, sorry), and the whole area of daemon management, logging, logrotation and so forth (not to mention buildout integration) seems not as advanced as we've come to know. What do people use here, what are your experiences and ideas? We've been using mod_wsgi for many moons. It's certainly very stable with piles of features for managing applications. Right now we're thinking about moving to Twisted because some bits of the application are asynchronous and already written on Twisted. It simplifies things to only have one daemon running everything. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.sqlalchemy release
On Mon, Dec 05, 2011 at 04:21:51PM -0500, Chris McDonough wrote: Tomorrow, I plan to: - Merge the chrism-py3 branch of zope.sqlalchemy into its trunk. (http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/) to get Python 3 compatibility. - Once the compat branch is merged, I'll make a 0.7 release of zope.sqlalchemy. Any dissent? None from me! This all looks like great work. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope] Python Script Security
Rich, You can modify the permissions on each of your python scripts to require some role, say gatekeeper and then proxy your front-end script to have the role gatekeeper. Then, only users or scripts with the role gatekeeper can call the script directly. Brian Brinegar Purdue University - Original Message - From: Richard Harley rich...@scholarpack.com To: zope@zope.org Sent: Friday, November 18, 2011 1:05:27 PM Subject: [Zope] Python Script Security Hello all I have a dtml method which builds a page from various different python scripts. How can I stop the python scripts from being called and passed variables independently of the main dtml method? I've tried some research on proxy roles but couldn't pin it down. ZSQL methods are not callable independently and I would like Python Scripts to behave the same. TIA Rich ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] zope-tests - FAILED: 8, OK: 39
On Sun, Nov 13, 2011 at 08:22:38AM -0500, Tres Seaver wrote: [8]FAILED winbot / z3c.form_py_265_32 https://mail.zope.org/pipermail/zope-tests/2011-November/052551.html Stephan said yesterday he thought this was a chameleon bug. I just tried to reproduce this on a Linux box with a fresh checkout, using Python 2.5.5. The tests don't even run for me:: Traceback (most recent call last): File /tmp/zft/bin/test, line 26, in module '--test-path', '/tmp/zft/src', File /tmp/zft/eggs/zope.testrunner-4.0.4-py2.5.egg/zope/testrunner/__init__.py, line 30, in run failed = run_internal(defaults, args, script_parts=script_parts) File /tmp/zft/eggs/zope.testrunner-4.0.4-py2.5.egg/zope/testrunner/__init__.py, line 41, in run_internal from zope.testrunner.runner import Runner File /tmp/zft/eggs/zope.testrunner-4.0.4-py2.5.egg/zope/testrunner/runner.py, line 46, in module import zope.testrunner.tb_format File /tmp/zft/eggs/zope.testrunner-4.0.4-py2.5.egg/zope/testrunner/tb_format.py, line 19, in module import zope.exceptions.exceptionformatter File /tmp/zft/eggs/zope.exceptions-3.6.1-py2.5.egg/zope/exceptions/__init__.py, line 24, in module import zope.security File /tmp/zft/eggs/zope.security-3.8.3-py2.5-linux-i686.egg/zope/security/__init__.py, line 17, in module from zope.security.management import checkPermission File /tmp/zft/eggs/zope.security-3.8.3-py2.5-linux-i686.egg/zope/security/management.py, line 20, in module from zope.security import interfaces File /tmp/zft/eggs/zope.security-3.8.3-py2.5-linux-i686.egg/zope/security/interfaces.py, line 19, in module from zope.schema import Text, TextLine File /tmp/zft/eggs/zope.schema-4.0.0-py2.5.egg/zope/schema/__init__.py, line 16, in module from zope.schema._field import Field, Container, Iterable, Orderable File /tmp/zft/eggs/zope.schema-4.0.0-py2.5.egg/zope/schema/_field.py, line 78 class SourceText(Text): ^ SyntaxError: invalid syntax I managed to reproduce it with Python 2.6. It turned out to be a bug I introduced with the zope.schema 4.0.0 Python 3 support. I've fixed it in 123340 and released as 4.0.1. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope-tests - FAILED: 1, OK: 45, UNKNOWN: 1
On Tue, Nov 15, 2011 at 01:00:03AM +, Zope tests summarizer wrote: Non-OK results -- [1]UNKNOWN UNKNOWN : Zope-trunk Python-2.6.6 : Linux https://mail.zope.org/pipermail/zope-tests/2011-November/052670.html Looks like a network failure [2]FAILED winbot / z3c.form_py_265_32 https://mail.zope.org/pipermail/zope-tests/2011-November/052649.html Today's run of this buildbot is green. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope] Importing Unauthorized?
We're in the process of migrating from Zope 2.11 to Zope 2.13, we have scripts which raise string exceptions, which is no longer allowed by the version of python. In particular, we have scripts which: raise Unauthorized However, we don't seem to be able to import the Unauthorized exception from AccessControl or zExceptions. What is the new way to raise an Unauthorized exception from within a Zope Python Script? Thanks, Brian Brinegar Web Services Coordinator Engineering Computer Network ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] zope.schema and Python 3
On Fri, Oct 14, 2011 at 12:52:00PM +0200, Brian Sutherland wrote: On Mon, Oct 10, 2011 at 01:14:26PM +0200, Brian Sutherland wrote: I've managed to port zope.schema to Python 3.2 on a branch (jinty-python3). It gives up Python 2.5 compatibility and depends on six (http://packages.python.org/six/). Any objections to me merging this branch? I just merged this to trunk. Thanks to some great work by Tres, it looks like the buildbots are green and this change is bedded down. Could someone release zope.schema trunk? (or give me rights to do so, jinty on pypi) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.schema and Python 3
On Mon, Nov 07, 2011 at 01:38:21PM -0500, Jim Fulton wrote: On Mon, Nov 7, 2011 at 1:09 PM, Brian Sutherland br...@vanguardistas.net wrote: On Fri, Oct 14, 2011 at 12:52:00PM +0200, Brian Sutherland wrote: On Mon, Oct 10, 2011 at 01:14:26PM +0200, Brian Sutherland wrote: I've managed to port zope.schema to Python 3.2 on a branch (jinty-python3). It gives up Python 2.5 compatibility and depends on six (http://packages.python.org/six/). Any objections to me merging this branch? I just merged this to trunk. Thanks to some great work by Tres, it looks like the buildbots are green and this change is bedded down. Could someone release zope.schema trunk? (or give me rights to do so, jinty on pypi) I empowered you! :) Drat, I forgot the unreleased Python 3 support in zope.testing. Could you empower me there too? -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope-tests - FAILED: 21, OK: 31
On Thu, Oct 20, 2011 at 11:58:09AM -0400, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [1]FAILED ZTK 1.0dev / Python2.4.6 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051353.html [2]FAILED ZTK 1.0dev / Python2.5.5 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051351.html [3]FAILED ZTK 1.0dev / Python2.6.5 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051352.html These are all failing due to a missing version pin for 'six', which should not happen: the ZTK 1.0 package set should not include the new version of 'zope.schema' which introduced this dependency. [4]FAILED ZTK 1.1 / Python2.5.5 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051398.html [5]FAILED ZTK 1.1 / Python2.6.5 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051397.html [6]FAILED ZTK 1.1 / Python2.7.2 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051396.html This failure is due to the unexpected presence of a 'Data.fs' file in what is supposed to be a newly-created tempdir. I don't understand how that is supposed to work, but the bug seems real. [7]FAILED ZTK 1.1dev / Python2.5.5 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051354.html Syntax errors due to Py3k compatibility changes in 'zope.schema'. The ZTK 1.1dev checkouts pull in the trunk of 'zope.schema', which is now backward-incompatible (it drops support for Python 2.5). I think we should be creating a '3.8' branch of 'zope.schema' (copied from 3.8.1), and make the ZTK 1.1dev checkout use that. Thanks for fixing these. I didn't realize that the buildouts would break because of my change. I should have though. (I've also given up looking at the buildout perma-failures a long time ago) Meanwhile, I think we should update the 'zope.schema' trunk to indicate that it is targeting a new major version, 4.0, due to new dependencies and dropping Python 2.5 compatibility. I've done this. [8]FAILED ZTK 1.1dev / Python2.6.5 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051356.html There is an odd 'zope.testrunner' failure here: Usage: test [options] [MODULE] [TEST] test: error: option --profile: invalid choice: 'cProfile' (choose from ) Failure in test /home/ccomb/ztk1.1dev-slave/Python2.6.5-Linux-64bit/build/src/zope.testrunner/src/zope/testrunner/testrunner-subunit.txt Failed doctest test for testrunner-subunit.txt And also the unexpected Data.fs issue which affected #4, 5, and 6. [9]FAILED ZTK 1.1dev / Python2.7.2 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-October/051355.html This is the unexpected Data.fs issue which affected #4, 5, and 6. I'm ignoring the 'z3c.*' failures: I still want these tests either fixed or else dropped from the daily report. [21] FAILED winbot / ztk_dev py_254_win32 https://mail.zope.org/pipermail/zope-tests/2011-October/051382.html This is the same Python 2.5 incompatibility problem which afflicts #7. This is a tricky issue, there are 2 parts: * Supporting python 2.5 in zope.schema is not hard, but will make the code quite ugly (no class decorator for implements) * I'm not sure if the next version of the ZTK will need to support Python 2.5 So there seem to be 3 options to fix this buildout issue: 1. Point the ZTK trunk at the zope.schema 3.8 branch 2. Make the zope.schema code uglier and support 2.5 3. Declare that the next ZTK version drops support for 2.5 and stop the buildbot for Python 2.5 + ZTK trunk I'm partial to option 1 to get the buildbot working and defer the Python 2.5/ugly code decision to later. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope] Manipulating images in Zope/Python
I am looking for strategy to programmatically create a customized printable certificate of completion for our Learning Delivery System courses (the system is Zope/Python based). My current plan is to use the Python Imaging Library to create the image (probably by overlaying some text on a stock image of a completion certificate to create a new image). The final image will then be emailed as an attachment to users. Is this a reasonable approach? Any others that anybody can suggest? Any samples of doing anything like this that anyone can point to? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] zope.schema and Python 3
On Mon, Oct 10, 2011 at 01:14:26PM +0200, Brian Sutherland wrote: I've managed to port zope.schema to Python 3.2 on a branch (jinty-python3). It gives up Python 2.5 compatibility and depends on six (http://packages.python.org/six/). Any objections to me merging this branch? I just merged this to trunk. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.schema and Python 3
On Tue, Oct 11, 2011 at 10:43:02AM +0200, Lennart Regebro wrote: On Tue, Oct 11, 2011 at 10:22, Brian Sutherland br...@vanguardistas.net wrote: On Mon, Oct 10, 2011 at 01:14:26PM +0200, Brian Sutherland wrote: I've managed to port zope.schema to Python 3.2 on a branch (jinty-python3). One doubt which has just crept up on me is if these classes: ASCII, ASCIILine, URI, Id, DottedName should still inherit from Bytes on Python 3. It seems more logical that they should inherit from Text. I had to do some gymnastics to keep them bytes on python 3 and feel all dirty about it. DottedName, in particular, describes itself as Python-style dotted names. But in Python 3 this code works: from 漢語 import Español print(Español) class '漢語.Español' so a DottedName could be '漢語.Español' under Python 3. Definitely unicode and not Bytes. However, changing the behaviour of these classes between Python 2 and 3 may be even more problematic. Another option is simply to have these classes raise NotImplementedError in their validate() methods under Python 3 and wait till the solution becomes more obvious. Anyone out there willing to assuage my doubts? It seems to me that they should be `str`, all of them, except possibly URI. Yeah, seems like it. I made this change to all except URI. I'm unsure exactly how that changes the behavior. ASCII/ASCIILine should have validators to restrict them to ASCII only, reasonably, I don't know if they already do. It has this: if not max(map(ord, value)) 128: raise InvalidValue which seems like it's enough. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] zope.schema and Python 3
I've managed to port zope.schema to Python 3.2 on a branch (jinty-python3). It gives up Python 2.5 compatibility and depends on six (http://packages.python.org/six/). Any objections to me merging this branch? -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Referring to same interface using zope.schema.Object
On Fri, Jul 22, 2011 at 04:29:32PM +0530, Joe Steeve wrote: Hello, I am trying to construct an object tree. Every node in the tree is of the same type. I am trying to achieve something like: class INode(Interface): parent = Object( title=uParent node, schema=INode ) children = List( title=u'Child nodes', value_type=Object(schema=INode) ) The above fails with NameError: name 'INode' is not defined. Any clues as to how to solve this? This would be my first guess: class INode(Interface): pass INode.parent = Object( title=uParent node, schema=INode ) INode.children = List( title=u'Child nodes', value_type=Object(schema=INode) ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Referring to same interface using zope.schema.Object
On Fri, Jul 22, 2011 at 01:41:32PM +0200, Jacob Holm wrote: On 2011-07-22 13:26, Brian Sutherland wrote: This would be my first guess: And that guess would be wrong. Thanks. Guess I should stop guessing;) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] z3c.form missing *.mo files?
On Tue, Jul 12, 2011 at 07:26:59PM +0300, Marius Gedminas wrote: On Tue, Jul 12, 2011 at 07:19:07PM +0300, Marius Gedminas wrote: And then what? Surely just having python-gettext installed and a couple of environment variables defined is not enough to magically produce .mo files Yes, actually it does ;) But the compiling happens during run-time when you start up the server. Neither the sdist nor binary egg need to contain the .mo files. I can't remember if it actually writes those files out to the filesystem. -- I would also need to run some Python script somewhere, presumably. Is it z3c.form's bin/pocompile? So I tried cd ~/src/z3c.form bin/pocompile python setup.py sdist and found that my dist/z3c.form-2.4.5dev.tar.gz now contains a .mo next to each .po in src/z3c/form/locales/. So, should I release z3c.form 2.4.5 with the .mo's? I would also like to add a RELEASING.txt with something like this If you're making a public z3c.form release to the Python Package Index, don't forget to generate .mo files with:: bin/pocompile before you do the usual ``python setup.py sdist register upload`` step. Ideally, I'd like ``python setup.py sdist`` itself to check for up-to-date .mo's and either regenerate them automatically, or at least complain loudly. I normally use a setuptools plugin I wrote for this: http://pypi.python.org/pypi/van.potomo The chicken and egg issue means it's probably worse than what you wrote. /me hoping that distutils2 will take care of these things. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-Checkins] SVN: Zope/trunk/ LP #787541: Fix WSGIPublisher to close requests on abort unconditionally.
Log message for revision 121832:
LP #787541: Fix WSGIPublisher to close requests on abort unconditionally.
Previously an addAfterCommitHook was used, but this is not run on transaction
aborts. Now a Synchronizer is used which unconditionally closes the request
after a transaction is finished.
Not closing the request seems to have caused the database connections to be
kept open and these nasty log messages:
CRITICAL ZODB.DB DB.open() has 42 open connections with a pool_size of 7
42 being the wrong answer in this case.
Changed:
U Zope/trunk/doc/CHANGES.rst
U Zope/trunk/src/ZPublisher/WSGIPublisher.py
U Zope/trunk/src/ZPublisher/tests/test_WSGIPublisher.py
-=-
Modified: Zope/trunk/doc/CHANGES.rst
===
--- Zope/trunk/doc/CHANGES.rst 2011-05-30 10:07:08 UTC (rev 121831)
+++ Zope/trunk/doc/CHANGES.rst 2011-05-30 11:47:48 UTC (rev 121832)
@@ -11,6 +11,11 @@
Bugs Fixed
++
+- LP #787541: Fix WSGIPublisher to close requests on abort unconditionally.
+ Previously an addAfterCommitHook was used, but this is not run on transaction
+ aborts. Now a Synchronizer is used which unconditionally closes the request
+ after a transaction is finished.
+
- Fix `WSGIResponse` and `publish_module` functions such that they
support the `IStreamIterator` interface in addition to `file` (as
supported by `ZServer.HTTPResponse`).
Modified: Zope/trunk/src/ZPublisher/WSGIPublisher.py
===
--- Zope/trunk/src/ZPublisher/WSGIPublisher.py 2011-05-30 10:07:08 UTC (rev
121831)
+++ Zope/trunk/src/ZPublisher/WSGIPublisher.py 2011-05-30 11:47:48 UTC (rev
121832)
@@ -200,6 +200,31 @@
return response
+class _RequestCloserForTransaction(object):
+Unconditionally close the request at the end of a transaction.
+
+See transaction.interfaces.ISynchronizer
+
+
+def __init__(self):
+self.requests = {}
+
+def add(self, txn, request):
+assert txn not in self.requests
+self.requests[txn] = request
+
+def beforeCompletion(self, txn):
+pass
+
+newTransaction = beforeCompletion
+
+def afterCompletion(self, txn):
+request = self.requests.pop(txn, None)
+if request is not None:
+request.close()
+
+_request_closer_for_repoze_tm = _RequestCloserForTransaction()
+
def publish_module(environ, start_response,
_publish=publish,# only for testing
_response_factory=WSGIResponse, # only for testing
@@ -214,6 +239,13 @@
response._server_version = environ.get('SERVER_SOFTWARE')
request = _request_factory(environ['wsgi.input'], environ, response)
+
+if 'repoze.tm.active' in environ:
+# NOTE: registerSynch is a no-op after the first request
+transaction.manager.registerSynch(_request_closer_for_repoze_tm)
+txn = transaction.get()
+_request_closer_for_repoze_tm.add(txn, request)
+
setDefaultSkin(request)
try:
@@ -237,10 +269,7 @@
# XXX This still needs verification that it really works.
result = (stdout.getvalue(), response.body)
-if 'repoze.tm.active' in environ:
-txn = transaction.get()
-txn.addAfterCommitHook(lambda ok: request.close())
-else:
+if 'repoze.tm.active' not in environ:
request.close() # this aborts the transation!
stdout.close()
Modified: Zope/trunk/src/ZPublisher/tests/test_WSGIPublisher.py
===
--- Zope/trunk/src/ZPublisher/tests/test_WSGIPublisher.py 2011-05-30
10:07:08 UTC (rev 121831)
+++ Zope/trunk/src/ZPublisher/tests/test_WSGIPublisher.py 2011-05-30
11:47:48 UTC (rev 121832)
@@ -414,6 +414,7 @@
def test_request_not_closed_when_tm_middleware_active(self):
import transaction
+from ZPublisher import WSGIPublisher
environ = self._makeEnviron()
environ['repoze.tm.active'] = 1
start_response = DummyCallable()
@@ -430,7 +431,22 @@
_request_factory=_request_factory)
self.assertFalse(_request._closed)
txn = transaction.get()
-self.assertTrue(list(txn.getAfterCommitHooks()))
+self.assertTrue(txn in
WSGIPublisher._request_closer_for_repoze_tm.requests)
+txn.commit()
+self.assertTrue(_request._closed)
+self.assertFalse(txn in
WSGIPublisher._request_closer_for_repoze_tm.requests)
+# try again, but this time raise an exception and abort
+_request._closed = False
+_publish._raise = Exception('oops')
+self.assertRaises(Exception, self._callFUT,
+ environ, start_response, _publish,
+ _request_factory=_request_factory)
+self.assertFalse(_request._closed)
+txn =
[Zope-Checkins] SVN: Zope/branches/2.13/ Merge r121832 from trunk to fix LP: #787541
Log message for revision 121833:
Merge r121832 from trunk to fix LP: #787541
Changed:
U Zope/branches/2.13/doc/CHANGES.rst
U Zope/branches/2.13/src/ZPublisher/WSGIPublisher.py
U Zope/branches/2.13/src/ZPublisher/tests/test_WSGIPublisher.py
-=-
Modified: Zope/branches/2.13/doc/CHANGES.rst
===
--- Zope/branches/2.13/doc/CHANGES.rst 2011-05-30 11:47:48 UTC (rev 121832)
+++ Zope/branches/2.13/doc/CHANGES.rst 2011-05-30 12:04:30 UTC (rev 121833)
@@ -11,6 +11,10 @@
Bugs Fixed
++
+- LP #787541: Fix WSGIPublisher to close requests on abort unconditionally.
+ Previously an addAfterCommitHook was used, but this is not run on transaction
+ aborts. Now a Synchronizer is used which unconditionally closes the request
+ after a transaction is finished.
Features Added
++
Modified: Zope/branches/2.13/src/ZPublisher/WSGIPublisher.py
===
--- Zope/branches/2.13/src/ZPublisher/WSGIPublisher.py 2011-05-30 11:47:48 UTC
(rev 121832)
+++ Zope/branches/2.13/src/ZPublisher/WSGIPublisher.py 2011-05-30 12:04:30 UTC
(rev 121833)
@@ -200,6 +200,31 @@
return response
+class _RequestCloserForTransaction(object):
+Unconditionally close the request at the end of a transaction.
+
+See transaction.interfaces.ISynchronizer
+
+
+def __init__(self):
+self.requests = {}
+
+def add(self, txn, request):
+assert txn not in self.requests
+self.requests[txn] = request
+
+def beforeCompletion(self, txn):
+pass
+
+newTransaction = beforeCompletion
+
+def afterCompletion(self, txn):
+request = self.requests.pop(txn, None)
+if request is not None:
+request.close()
+
+_request_closer_for_repoze_tm = _RequestCloserForTransaction()
+
def publish_module(environ, start_response,
_publish=publish,# only for testing
_response_factory=WSGIResponse, # only for testing
@@ -214,6 +239,13 @@
response._server_version = environ.get('SERVER_SOFTWARE')
request = _request_factory(environ['wsgi.input'], environ, response)
+
+if 'repoze.tm.active' in environ:
+# NOTE: registerSynch is a no-op after the first request
+transaction.manager.registerSynch(_request_closer_for_repoze_tm)
+txn = transaction.get()
+_request_closer_for_repoze_tm.add(txn, request)
+
setDefaultSkin(request)
try:
@@ -237,10 +269,7 @@
# XXX This still needs verification that it really works.
result = (stdout.getvalue(), response.body)
-if 'repoze.tm.active' in environ:
-txn = transaction.get()
-txn.addAfterCommitHook(lambda ok: request.close())
-else:
+if 'repoze.tm.active' not in environ:
request.close() # this aborts the transation!
stdout.close()
Modified: Zope/branches/2.13/src/ZPublisher/tests/test_WSGIPublisher.py
===
--- Zope/branches/2.13/src/ZPublisher/tests/test_WSGIPublisher.py
2011-05-30 11:47:48 UTC (rev 121832)
+++ Zope/branches/2.13/src/ZPublisher/tests/test_WSGIPublisher.py
2011-05-30 12:04:30 UTC (rev 121833)
@@ -414,6 +414,7 @@
def test_request_not_closed_when_tm_middleware_active(self):
import transaction
+from ZPublisher import WSGIPublisher
environ = self._makeEnviron()
environ['repoze.tm.active'] = 1
start_response = DummyCallable()
@@ -430,7 +431,22 @@
_request_factory=_request_factory)
self.assertFalse(_request._closed)
txn = transaction.get()
-self.assertTrue(list(txn.getAfterCommitHooks()))
+self.assertTrue(txn in
WSGIPublisher._request_closer_for_repoze_tm.requests)
+txn.commit()
+self.assertTrue(_request._closed)
+self.assertFalse(txn in
WSGIPublisher._request_closer_for_repoze_tm.requests)
+# try again, but this time raise an exception and abort
+_request._closed = False
+_publish._raise = Exception('oops')
+self.assertRaises(Exception, self._callFUT,
+ environ, start_response, _publish,
+ _request_factory=_request_factory)
+self.assertFalse(_request._closed)
+txn = transaction.get()
+self.assertTrue(txn in
WSGIPublisher._request_closer_for_repoze_tm.requests)
+txn.abort()
+self.assertFalse(txn in
WSGIPublisher._request_closer_for_repoze_tm.requests)
+self.assertTrue(_request._closed)
class DummyRequest(dict):
___
Zope-Checkins maillist - Zope-Checkins@zope.org
https://mail.zope.org/mailman/listinfo/zope-checkins
Re: [Zope-dev] zope-tests - FAILED: 14, OK: 74, UNKNOWN: 3
On Thu, Apr 21, 2011 at 01:00:20AM -0400, Zope tests summarizer wrote: [13] FAILED winbot / ztk_dev py_254_win32 https://mail.zope.org/pipermail/zope-tests/2011-April/038623.html [14] FAILED winbot / ztk_dev py_265_win32 https://mail.zope.org/pipermail/zope-tests/2011-April/038624.html [15] FAILED winbot / ztk_dev py_265_win64 https://mail.zope.org/pipermail/zope-tests/2011-April/038625.html [16] FAILED winbot / ztk_dev py_270_win32 https://mail.zope.org/pipermail/zope-tests/2011-April/038626.html [17] FAILED winbot / ztk_dev py_270_win64 https://mail.zope.org/pipermail/zope-tests/2011-April/038627.html These are hopefully fixed in 121453. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Tests: 74 OK, 14 Failed
On Tue, Mar 29, 2011 at 10:22:45AM -0400, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Subject: FAILED : Zope Buildbot / zopetoolkit-1.0_win-py2.6 slave-win From: jdriessen at thehealthagency.com Date: Mon Mar 28 15:18:14 EDT 2011 URL: http://mail.zope.org/pipermail/zope-tests/2011-March/036429.html Subject: FAILED : Zope Buildbot / zope2.12-py2.6 slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Mon Mar 28 16:46:58 EDT 2011 URL: http://mail.zope.org/pipermail/zope-tests/2011-March/036443.html Subject: FAILED : Zope Buildbot / zope2.13-py2.6 slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Mon Mar 28 16:47:05 EDT 2011 URL: http://mail.zope.org/pipermail/zope-tests/2011-March/036444.html Subject: FAILED : Zope Buildbot / zopetoolkit-1.0-py2.6 slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Mon Mar 28 17:06:20 EDT 2011 URL: http://mail.zope.org/pipermail/zope-tests/2011-March/036453.html These four are still dying in the bootstrap step, without any kind of useful output. I just tried to reproduce one without success:: - --- % -- $ cd /tmp $ svn co http://svn.zope.org/repos/main/zopetoolkit/branches/1.0 A1.0/LICENSE.txt A1.0/development.cfg A1.0/zopeapp-versions.cfg A1.0/bootstrap.py A1.0/buildout.cfg A1.0/COPYRIGHT.txt A1.0/ztk.cfg A1.0/README.txt A1.0/zopeapp.cfg A1.0/ztk-versions.cfg A1.0/index.rst U 1.0 Checked out revision 121155. $ cd 1.0/ $ /opt/Python-2.6.5/bin/python bootstrap.py Creating directory '/tmp/1.0/bin'. Creating directory '/tmp/1.0/parts'. Creating directory '/tmp/1.0/eggs'. Creating directory '/tmp/1.0/develop-eggs'. Generated script '/tmp/1.0/bin/buildout'. - --- % -- I also couldn't reproduce the failure. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Wed, Mar 16, 2011 at 12:08:12AM +0100, Jan-Jaap Driessen wrote: On 10 March 2011 18:18, Brian Sutherland br...@vanguardistas.net wrote: On Thu, Mar 10, 2011 at 07:07:24PM +0200, Marius Gedminas wrote: On Thu, Mar 10, 2011 at 04:22:25PM +0100, Brian Sutherland wrote: I now believe I've taken care of all the issues raised in your review. connection.py still feels uncomfortable, but I can live with it Unless there are any objections, I plan to merge all three branches (zope.testbrowser, zope.app.wsgi and zope.app.testing) early next week. Is there anyone willing to make releases of these to PyPI? (or give me the access to do it myself) I can give you PyPI access to all three. Great! Note: when asking for PyPI access it's customary to mention one's PyPI username. I hadn't used it for such a long time I even had to look it up: jinty -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) Just released zope.app.wsgi 3.13 that picks up the changes in zope.testbrowser 4.0, also updated ZTK trunk. Great, it looks like we're done. Aside from keeping an eye on the buildbots, of course. Thanks for all the help! -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Sat, Mar 05, 2011 at 01:07:58AM +0100, Jan-Jaap Driessen wrote: To get the groktoolkit tests and my company's applications to run with Brian's zope.testbrowser+webtest branch I did the following things: * I created a branch of zope.app.wsgi trunk [1] to which I copied the AuthorizationMiddleware from zope.testbrowser trunk (moved there in [2]). The three middleware components (Transaction, Authorization and HandleErrors) are needed to make the groktoolkit tests pass. The AuthorizationMiddleware is not available on Brian's branch, but is available on zope.testbrowser trunk. If we can land this middleware in zope.testbrowser, we can again remove it from zope.app.wsgi.testlayer. In my opinion, this middleware should be part of zope.testbrowser, because handleErrors is part of the IBrowser interface. * The `http caller` in zope.app.wsgi.testlayer is used in grokcore.rest. On my branch, I rewrote the http caller to use WebTest and _APP_UNDER_TEST instead of wsgi_intercept. The import from zope.testbrowser is bothering me; can we put the http caller in zope.testbrowser? The http caller returns a FakeResponse instance, which could very well be a WebTest TestResponse. The FakeResponse wrapper is merely used to fix the current tests. * I fixed some tests in grok and grokcore.rest to use the WebTest-style http caller. I was pretty sure you wouldn't mind so I committed a few cleanups and bugfixes to your zope.app.wsgi branch. The trunk ztk and zopeapp tests now also pass with the three branches. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Thu, Mar 10, 2011 at 07:07:24PM +0200, Marius Gedminas wrote: On Thu, Mar 10, 2011 at 04:22:25PM +0100, Brian Sutherland wrote: I now believe I've taken care of all the issues raised in your review. connection.py still feels uncomfortable, but I can live with it Unless there are any objections, I plan to merge all three branches (zope.testbrowser, zope.app.wsgi and zope.app.testing) early next week. Is there anyone willing to make releases of these to PyPI? (or give me the access to do it myself) I can give you PyPI access to all three. Great! Note: when asking for PyPI access it's customary to mention one's PyPI username. I hadn't used it for such a long time I even had to look it up: jinty -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Sat, Mar 05, 2011 at 01:07:58AM +0100, Jan-Jaap Driessen wrote: Hello from the grok department, On 4 March 2011 11:48, Brian Sutherland br...@vanguardistas.net wrote: On Fri, Mar 04, 2011 at 10:25:59AM +0100, Sylvain Viollon wrote: Hello, On Fri, 4 Mar 2011 10:15:18 +0100 Brian Sutherland br...@vanguardistas.net wrote: In zope.app.wsgi all those idiosyncracies are more or less handled by a WSGI middleware. I guess you can reuse it. One of the purpose of the zope.app.wsgi implementation was to be able to convert the existing tests just by changing the import, and get ride of zope.testbrowser dependencies on the crazy zope.app.testing nobody-knows-what-it-does-and-does-everything in Grok (without adding many others). Which worked perfectly. I'll definitely look into reusing the middleware. But, tell me, how do I run all the grok tests? (So I can see what idiosyncracies are required) There is a Grok ToolKit, that works exactly like the Zope ToolKit: http://svn.zope.org/groktoolkit/trunk If you run this buildout you should get a ./bin/test-grok command that runs all the tests, using z3c.recipe.compattest. Actually, I think you can start by testing only grokcore.view. I guess it is one of the base packages that uses a lot the test browser. Ok, there are 2 grokcore.view failures with my branch. The first is due to the Basic Auth header and is fixable by adding the middleware we were talking about. To get the groktoolkit tests and my company's applications to run with Brian's zope.testbrowser+webtest branch I did the following things: Great, thanks a lot!!! * I created a branch of zope.app.wsgi trunk [1] to which I copied the AuthorizationMiddleware from zope.testbrowser trunk (moved there in [2]). The three middleware components (Transaction, Authorization and HandleErrors) are needed to make the groktoolkit tests pass. The AuthorizationMiddleware is not available on Brian's branch I've added the AuthorizationMiddleware to zope.testbrowser.wsgi and added some tests for it. It doesn't look like it'll be much of a maintenance headache being very simple with no dependencies. , but is available on zope.testbrowser trunk. If we can land this middleware in zope.testbrowser, we can again remove it from zope.app.wsgi.testlayer. In my opinion, this middleware should be part of zope.testbrowser, because handleErrors is part of the IBrowser interface. Hmm, the TransactionMiddleware looks very app-specific, as does the HandleErrorsMiddleware. In fact, I think you'll find that with my branch the HandleErrorsMiddleware is a no-op and can be removed. The 'wsgi.handleErrors' environment variable is set in zope.testbrowser.wsgi directly now. * The `http caller` in zope.app.wsgi.testlayer is used in grokcore.rest. On my branch, I rewrote the http caller to use WebTest and _APP_UNDER_TEST instead of wsgi_intercept. The import from zope.testbrowser is bothering me; can we put the http caller in zope.testbrowser? The http caller returns a FakeResponse instance, which could very well be a WebTest TestResponse. The FakeResponse wrapper is merely used to fix the current tests. I see. I think the layer just needs to become better about exposing the global application under test. I've just committed a patch to make the .app property a get_app classmethod. So you can do: global_app_under_test = zope.testbrowser.wsgi.Layer.get_app() rather than access _APP_UNDER_TEST directly. I'd rather not move the http caller to zope.testbrowser as I'm not sure what functionality it provides over just using webtest.TestApp directly. * I fixed some tests in grok and grokcore.rest to use the WebTest-style http caller. The other is more interesting, in src/grokcore/view/ftests/url/redirect.py. It seems that that test is actually making calls over the network to google's servers. Probably not a good thing. The new behaviour (probably not good either) is to send the request to the application even if the hostname is www.google.com. I would guess that the right behaviour would be to raise an error if the hostname is not localhost or 127.0.0.1. +1 Ok, committed this. I've also allowed domains like test.example.com. In order to run the tests on the groktoolkit trunk, run: bin/buildout sources:zope.testbrowser=svn svn+ssh://svn.zope.org/repos/main/zope.testbrowser/branches/jinty-webtest3 sources:zope.app.wsgi=svn svn+ssh://svn.zope.org/repos/main/zope.app.wsgi/branches/janjaapdriessen-webtest buildout:auto-checkout=zope.testbrowser zope.app.wsgi grok grokcore.rest versions:WebTest=1.2.3 bin/test-grok -- Jan-Jaap Driessen 1) http://zope3.pov.lt/trac/changeset/120756/zope.app.wsgi/branches/janjaapdriessen-webtest/src/zope/app/wsgi/testlayer.py 2) http://zope3.pov.lt/trac/changeset/119878/zope.app.wsgi/trunk/src/zope/app/wsgi/testlayer.py -- Brian
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Wed, Mar 02, 2011 at 09:16:14AM +0100, Wolfgang Schnerring wrote: Hello Brian, it's taken a while, but I finally had a chance to review your branch(es). Great! * Brian Sutherland br...@vanguardistas.net [2011-02-12 18:57]: On Tue, Feb 01, 2011 at 09:32:11AM +0100, Wolfgang Schnerring wrote: I'd prefer if we treated this as two separate steps, then: a) improve the testbrwoser+wsgi story by replacing wsgi_intercept with WebTest I pulled this out of my original branch and put it here: svn+ssh://svn.zope.org/repos/main/zope.testbrowser/jinty-webtest3-minimal Thanks, that helped me understand what's going on much better. I do have a few questions about this part: - Does the (webtest-based) wsgi.Browser behave similarly to the Publisher-Browser? When we ported the wsgi_intercept variant from zope.app.wsgi, we found that there were some idiosyncracies woven in that people may rely on. This kind of stuff of course isn't documented, and we didn't do much research, but rather took care to port what we found in zope.app.wsgi, namely filtering some HTTP headers from the response (meh, doesn't feel important), and handling Basic Auth headers (that feels important to preserve). I feel strongly that these idiosyncrasies shouldn't get built into the new WSGI testbrowser and perpetuated. You had some WSGI middleware which implemented these. I think we can do it in the same way and anyone migrating from Publisher-Browser based code could use it. Perhaps that needs to be inside zope.testbrowser itself, or could be done in zope.app.wsgi. Depends on the details I guess. Unfortunately, as you say, they are undocumented and seem to be untested. I would like to make a deeper investigation of those issues and write some tests. Is there any test suite (grok perhaps?) that I can run using the new WSGI based browser that'll highlight these issues? - What should happen in zope.app.wsgi? We moved the wsgi-based Testbrowser from there and left BBB imports to zope.testbrowser.wsgi, taking care to be API compatible. I guess that won't be possible with the WebTest-based Testbrowser (or would it?), so we probably have to break that. (Hmm, would that imply a major version bump there, too?) Since the Grok people are the ones who probably use the zope.app.wsgi Testbrowser the most, they should probably take the WebTest-Testbrowser for a test drive and see whether it works for them, otherwise we're going to make them quite unhappy if we just break zope.app.wsgi. (Could you ask on grok-dev about that? I guess it's too buried here to be noticed.) I'll try to co-ordinate with the grok/zope.app.wsgi developers on this. I was unaware that they had BBB imports from zope.testbrowser. - What's connection.py? I don't really understand where that came from or what its purpose is, especially since wsgi.py seems to be the only one that uses it? Ah. I take that last bit back, the extracted Publisher-Browser in zope.app.testing uses it, so I guess this is a split-off artifact of the refactoring/extraction. But I still don't really understand what it is all about. %-) It's about minimizing the duplicated code in zope.app.testing.testbrowser and zope.testbrowser.wsgi. It's kind of reusable pieces to build zope.testbrowser connections. That said, I'm uncomfortable with it. connection.py is probably not really very reusable. And anyone changing it would have to be aware that it was reused in zope.app.testing. Another way would be to fold connection.py into zope.testbrowser.wsgi and copy the necessary code to zope.app.testing. It's not obvious to me which way is better. You could easily persuade me another way is better. - The layer looks good. (OK, so that's not a question ;) I'm glad you found a way that the browser can get the application out of the air. I've tried the explicit passing way on a project of mine, and it's a huge hassle (I've ended up stuffing a preconfigured browser instance into the doctest globs, ugh.) yeah, in the end wsgi_intercept uses global variables. So it's the same approach. I think it would be good if the layer stored the application of self.app, we did that in the wsgi_intercept variant and were glad of it a few times already. Sure, I've added that. b) extract the testbrowser part that talks to the Publisher This is here and by necessity includes the changes for step (a): svn+ssh://svn.zope.org/repos/main/zope.testbrowser/jinty-webtest3 svn+ssh://svn.zope.org/repos/main/zope.app.testing/branches/jinty-testbrowser The extraction of the Publisher-Browser makes a lot of sense, and looks clean to me, as does the rewrite of the tests to use a plain WSGI app instead of a Zope-based app. Great! I would much prefer to merge both steps together. Yes, I guess that makes sense because only step (b) includes proper tests for everything. Perfect. Wolfgang
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Thu, Mar 03, 2011 at 11:04:12AM +0100, Sylvain Viollon wrote: On Wed, 02 Mar 2011 09:16:14 +0100 Wolfgang Schnerring w...@gocept.com wrote: Hello Brian, Hello, it's taken a while, but I finally had a chance to review your branch(es). * Brian Sutherland br...@vanguardistas.net [2011-02-12 18:57]: On Tue, Feb 01, 2011 at 09:32:11AM +0100, Wolfgang Schnerring wrote: I'd prefer if we treated this as two separate steps, then: a) improve the testbrwoser+wsgi story by replacing wsgi_intercept with WebTest I pulled this out of my original branch and put it here: svn+ssh://svn.zope.org/repos/main/zope.testbrowser/jinty-webtest3-minimal Thanks, that helped me understand what's going on much better. I do have a few questions about this part: - Does the (webtest-based) wsgi.Browser behave similarly to the Publisher-Browser? When we ported the wsgi_intercept variant from zope.app.wsgi, we found that there were some idiosyncracies woven in that people may rely on. This kind of stuff of course isn't documented, and we didn't do much research, but rather took care to port what we found in zope.app.wsgi, namely filtering some HTTP headers from the response (meh, doesn't feel important), and handling Basic Auth headers (that feels important to preserve). Don't forget the support to do handleErrors = False. It is I think one of the most important, which help to debug the tests. That's supported for both zope.app.wsgi and Paste applications. YMMV for other WSGI applications. In zope.app.wsgi all those idiosyncracies are more or less handled by a WSGI middleware. I guess you can reuse it. One of the purpose of the zope.app.wsgi implementation was to be able to convert the existing tests just by changing the import, and get ride of zope.testbrowser dependencies on the crazy zope.app.testing nobody-knows-what-it-does-and-does-everything in Grok (without adding many others). Which worked perfectly. I'll definitely look into reusing the middleware. But, tell me, how do I run all the grok tests? (So I can see what idiosyncracies are required) Regards, Sylvain, -- Sylvain Viollon -- Infrae t +31 10 243 7051 -- http://infrae.com Hoevestraat 10 3033GC Rotterdam -- The Netherlands ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Fri, Mar 04, 2011 at 10:25:59AM +0100, Sylvain Viollon wrote: Hello, On Fri, 4 Mar 2011 10:15:18 +0100 Brian Sutherland br...@vanguardistas.net wrote: In zope.app.wsgi all those idiosyncracies are more or less handled by a WSGI middleware. I guess you can reuse it. One of the purpose of the zope.app.wsgi implementation was to be able to convert the existing tests just by changing the import, and get ride of zope.testbrowser dependencies on the crazy zope.app.testing nobody-knows-what-it-does-and-does-everything in Grok (without adding many others). Which worked perfectly. I'll definitely look into reusing the middleware. But, tell me, how do I run all the grok tests? (So I can see what idiosyncracies are required) There is a Grok ToolKit, that works exactly like the Zope ToolKit: http://svn.zope.org/groktoolkit/trunk If you run this buildout you should get a ./bin/test-grok command that runs all the tests, using z3c.recipe.compattest. Actually, I think you can start by testing only grokcore.view. I guess it is one of the base packages that uses a lot the test browser. Ok, there are 2 grokcore.view failures with my branch. The first is due to the Basic Auth header and is fixable by adding the middleware we were talking about. The other is more interesting, in src/grokcore/view/ftests/url/redirect.py. It seems that that test is actually making calls over the network to google's servers. Probably not a good thing. The new behaviour (probably not good either) is to send the request to the application even if the hostname is www.google.com. I would guess that the right behaviour would be to raise an error if the hostname is not localhost or 127.0.0.1. Regards, Sylvain, -- Sylvain Viollon -- Infrae t +31 10 243 7051 -- http://infrae.com Hoevestraat 10 3033GC Rotterdam -- The Netherlands -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope] ZODB Key error in root Zope database
We've got a bit of a situation...
We run Zope 2.11 with a main ZODB and a few dozen separate file storages
for sub-sites.
Occasionally, we've encountered the problem where a cut-and-paste occurs
between storages and upon a pack results in a key error. Normally, we
avoid this because the FileStorages represent individual sites.
Last night something happened where a script was cut-and-pasted from one
of the sub-storages into the main, root-level, database.
Upon a restart we encountered a key error:
File /var/local/zope/Zope-2.11.4-1/lib/python/ZODB/Connection.py,
line 361, in get_connection
new_con = self._db.databases[database_name].open(
KeyError: 'site1'
Normally, we've encountered these errors when a database is packed and
the solution is to revert to a pre-pack database and remove or fix the
offending object.
In this case, we can't even start Zope or mount the storage in a script
to remove the offending items.
Before I go on, I found that ZODB 3.9 provides options to disallow
cross-database references and we'll work to get to this version during
our next upgrade cycle.
In the mean time we've reverted to a previous backup prior to the
offending object being placed in the root database. This resulted in
about 200 MB of data loss. I would like a way to mount the corrupted
database and attempt to determine exactly what was lost.
What we've tried so far has gone like this:
from ZODB import FileStorage, DB
from OFS.Folder import Folder
storage = FileStorage.FileStorage('BadData.fs')
db = DB(storage)
conn = db.open()
root = conn.root()
app = root['Application']
app.manage_deleteObjects(['OffendingObject'])
# This throws the same exception so we tried working around the
# mechanics of manage_deleteObjects with this.
app._objects = tuple([i for i in app._objects if i['id'] !=
'OffendingObject'])
Same exception. Can't access the database. We've tried using _setOb to
replace the object.
Any suggestions? It seems as though it needs to be able to access
_objects before it mounts the mounted databases.
Thanks,
--
Brian Brinegar
Web Services Coordinator
Engineering Computer Network
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZODB Key error in root Zope database
FYI,
I was able to successfully remove the cross storage reference in the
root of my main Zope database by explicitly setting the missing database
connection to a hacked DummyConnection instance. I probably could have
dug deeper to create a real connection, but this worked.
from ZODB import FileStorage, DB
import transaction
storage = FileStorage.FileStorage('Data.fs')
app = DB(storage).open().root()['Application']
base_conn = app._p_jar
class DummyConnection:
implements enough of a connection to allow for
removing of the cross database reference
def __init__(self):
self.connections = []
self._cache = {}
def get(self, key):
return None
def open(self, **kw):
return DummyConnection()
base_conn._db.databases['site1'] = DummyConnection()
app._objects = tuple([i for i in app._objects if i['id'] != 'badID'])
transaction.get().commit()
It's quite the hack, but allowed us to get the database back.
Brian Brinegar
Web Services Coordinator
Engineering Computer Network
--
Brian Brinegar
Web Services Coordinator
Engineering Computer Network
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] Sprints at PyCon
On Thu, Feb 10, 2011 at 10:46:42AM -0500, Jim Fulton wrote: - Better integration between webob and ztk. Yay! -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Sat, Feb 12, 2011 at 02:59:50PM +0100, Wolfgang Schnerring wrote: Hi, sorry I'm replying this late. I've been busy, then sick, then busy again, so I'm afraid this has got pushed low on my stack of stuff... No worries, I'm not in a big rush :) * Brian Sutherland br...@vanguardistas.net [2011-02-02 11:15]: On Tue, Feb 01, 2011 at 09:32:11AM +0100, Wolfgang Schnerring wrote: * Brian Sutherland br...@vanguardistas.net [2011-01-31 09:54]: I'd prefer if we treated this as two separate steps, then: a) improve the testbrwoser+wsgi story by replacing wsgi_intercept with WebTest I pulled this out of my original branch and put it here: svn+ssh://svn.zope.org/repos/main/zope.testbrowser/jinty-webtest3-minimal The test coverage on this branch leaves much to be desired. b) extract the testbrowser part that talks to the Publisher This is here and by necessity includes the changes for step (a): svn+ssh://svn.zope.org/repos/main/zope.testbrowser/jinty-webtest3 svn+ssh://svn.zope.org/repos/main/zope.app.testing/branches/jinty-testbrowser I would much prefer to merge both steps together. Initially I did them together because it was the only way to get very good test coverage of the WebTest integration. If we do it this way, between steps a and b we'll have poor coverate. But that's not so bad as the code has already been well tested on my current branch. As to (a), I'll still need to look at your code, but as I said I'm all in favour of using WebTest instead of wsgi_intercept. I'll try make a branch for this in the next week or so for you to review. Thanks very much, I'll look throught it as soon as I find some time, probably not next week (I'm travelling), but hopefully the week after that, at the latest. Sure, I can wait a few weeks. But let me know if you're not going to get around to it. As to (b), I saw you moved the code to zope.app.testing. I have a few ideas in that area which I'll contact you off-list about. Sure! As we've discussed off-list, we now both feel that moving the publisher-testbrowser to zope.app.testing is a good idea, since the only applications that will want to use that testbrowser are those that use zope.app.testing -- the more modern approach of testing publisher-based applications is to use the code from zope.app.wsgi to create a WSGI application and then use the WSGI-enabled testbrowser to talk to that. Great, looks like we have a consensus :) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Tue, Feb 01, 2011 at 09:32:11AM +0100, Wolfgang Schnerring wrote: * Brian Sutherland br...@vanguardistas.net [2011-01-31 09:54]: On Mon, Jan 31, 2011 at 07:02:35AM +0100, Wolfgang Schnerring wrote: I'd very much like there to be *one* way of doing WSGI with the testbrowser, and at first blush I don't care whether that's WebTest or wsgi_intercept or whathaveyou, as long as it fulfills its purpose. We have already committed to wsgi_intercept integration for the long term. It was released in zope.testbrowser 3.11 a few days ago, right? If we are to have only one way to do this, then wsgi_intercept must be it. I definitely don't see this as set in stone, *at all*. Yes, we've had a release that uses wsgi_intercept for talking to WSGI apps. (And yes, we didn't ask anyone for their opinion on it, and I'm sorry about that. However, I guess the development process in the Zope community is an entirely different issue, sigh.) Yeah, a totally different issue that I don't really wanna get involved in right now. But I feel the important point about this regarding compatibility is not the underlying technology, but the API, i. e. that - zope.testbrowser.wsgi.Browser is a Testbrowser - zope.testbrowser.wsgi.Layer with a method make_wsgi_app is there to facilitate the test setup. I think as long as we preserve this API (which seems sound to me, but of course I'm biased ;-), we're free to change stuff under the hood. Ok, I think that that API is preservable. However I would like to make the layer optional by adding an wsgi_app keyword argument to Browser. So you can do: browser = Browser(wsgi_app=app) where app is a WSGI application. Some people in the non-zope world are not too fond of layers, or use incompatible testrunners. We shouldn't force layers on them. But I'm ambivalent about only having one way to do things. I think having both integrations in zope.testbrowser is not such a bad thing. Having the test suite run over 2 different integrations is definitely good. Maybe I'm missing something, but when I think about the task at hand from the client's perspective, then the only sentence that matters to me is, give me a zope.testbrowser that talks to this WSGI callable. Which means a) I couldn't care less how that's done internally as long as it doesn't cause me any hassle and, more importantly b) I do *not* want to have to think about it and/or make a choice about the underlying mechanism. I want the library to have done that research (as we are doing right now). And to me this task seems straightforward enough to not warrant pluggability -- on the contrary, I feel it's so narrow in scope it outright *forbids* it (but again, I may be missing something). Well, it appears that now we've reached some sort of consensus as to what base library to use. So yeah, having *one* way to do things is just fine. I'll gladly review your branch, but I'd like to know the motivation behind it. Only ~30% of the branch is the implementation of the WebTest connection. The other ~70% of the branch is a refactoring of the test suite. That's where the reduction in test dependencies comes from. The test fixture on the branch is a reasonably minimal WSGI application run via the WebTest connection. I'd prefer if we treated this as two separate steps, then: a) improve the testbrwoser+wsgi story by replacing wsgi_intercept with WebTest b) extract the testbrowser part that talks to the Publisher Initially I did them together because it was the only way to get very good test coverage of the WebTest integration. If we do it this way, between steps a and b we'll have poor coverate. But that's not so bad as the code has already been well tested on my current branch. As to (a), I'll still need to look at your code, but as I said I'm all in favour of using WebTest instead of wsgi_intercept. I'll try make a branch for this in the next week or so for you to review. As to (b), I saw you moved the code to zope.app.testing. I have a few ideas in that area which I'll contact you off-list about. Sure! Wolfgang ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest (round 2)
On Mon, Jan 31, 2011 at 07:02:35AM +0100, Wolfgang Schnerring wrote: * Brian Sutherland br...@vanguardistas.net [2011-01-30 16:04]: I've finally finished refactoring my WebTest/testbrowser branches, basically doing this: - Integrate with WebTest. zope.testbrowser.webtest.Browser is a new Browser implementation that uses webtest.TestApp to drive a WSGI application. This allows simple and direct testing of WSGI applications. - Re-write the test application as a pure WSGI application using WebOb. Run the existing tests using the WebTest based Browser - Move zope.app.testing based Browser into zope.app.testing (leaving backwards compatibility imports in-place). This is a very big change, so I would appreciate anyone who would take a look at these branches before I merge: Michael Howitz and I recently polished the integration of zope.testbrowser and wsgi_intercept to accomplish pretty much the same things you mentioned. (I'm aware that you two exchanged some emails about it, but don't know any details). So I'm curious: What are the differences bewteen WebTest and wsgi_intercept? Is one preferable to the other? The differences I've seen between the two libraries: * WebTest doesn't require a global setup/teardown. (no zope.testing Layer dependency) * With WebTest it's more difficult to accidentally end up sending real HTTP requests over the internet. I'm probably missing quite a few though. Another interesting datapoint is the number of downloads of each package from pypi.python.org: * http://pypi.python.org/pypi/wsgi_intercept/0.4 13000 downloads in 2 years * http://pypi.python.org/pypi/WebTest/1.2.3 18500 downloads in 2 months I'd very much like there to be *one* way of doing WSGI with the testbrowser, and at first blush I don't care whether that's WebTest or wsgi_intercept or whathaveyou, as long as it fulfills its purpose. We have already committed to wsgi_intercept integration for the long term. It was released in zope.testbrowser 3.11 a few days ago, right? If we are to have only one way to do this, then wsgi_intercept must be it. But I'm ambivalent about only having one way to do things. I think having both integrations in zope.testbrowser is not such a bad thing. Having the test suite run over 2 different integrations is definitely good. I'll gladly review your branch, but I'd like to know the motivation behind it. Only ~30% of the branch is the implementation of the WebTest connection. The other ~70% of the branch is a refactoring of the test suite. That's where the reduction in test dependencies comes from. The test fixture on the branch is a reasonably minimal WSGI application run via the WebTest connection. Attached is a patch for my branch to run the test suite with the wsgi_intercept integration. I havn't committed it yet as it has multiple failures. A way forward may be to fix the wsgi_intercept integration so that it can run the full test suite then merge my branch without the WebTest integration. However I would prefer to merge the WebTest integration as is and fix the wsgi_intercept integration. Wolfgang ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland Index: tests/test_doctests.py === --- tests/test_doctests.py (revision 120009) +++ tests/test_doctests.py (working copy) @@ -17,26 +17,45 @@ import zope.testbrowser.ftests.wsgitestapp import zope.testbrowser.webtest +import zope.testbrowser.wsgi -def make_browser(*args, **kw): +def make_webtest_browser(*args, **kw): app = zope.testbrowser.ftests.wsgitestapp.WSGITestApplication() return zope.testbrowser.webtest.Browser(app, *args, **kw) + +class WsgiInterceptLayer(zope.testbrowser.wsgi.Layer): + +def make_wsgi_app(self): +return zope.testbrowser.ftests.wsgitestapp.WSGITestApplication() + + def test_suite(): flags = doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS -suite = doctest.DocFileSuite( +webtest_suite = doctest.DocFileSuite( 'README.txt', 'cookies.txt', 'wsgi.txt', 'fixed-bugs.txt', optionflags=flags, -globs=dict(Browser=make_browser), +globs=dict(Browser=make_webtest_browser), checker=zope.testbrowser.tests.helper.checker, package='zope.testbrowser') +wsgi_suite = doctest.DocFileSuite( +'README.txt', +'cookies.txt', +'wsgi.txt', +'fixed-bugs.txt', +optionflags=flags, +globs=dict(Browser=zope.testbrowser.wsgi.Browser), +checker=zope.testbrowser.tests.helper.checker
[Zope-dev] zope.testbrowser and WebTest (round 2)
I've finally finished refactoring my WebTest/testbrowser branches, basically doing this: - Integrate with WebTest. zope.testbrowser.webtest.Browser is a new Browser implementation that uses webtest.TestApp to drive a WSGI application. This allows simple and direct testing of WSGI applications. - Re-write the test application as a pure WSGI application using WebOb. Run the existing tests using the WebTest based Browser - Move zope.app.testing based Browser into zope.app.testing (leaving backwards compatibility imports in-place). This refactoring removes these test dependencies from zope.testbroswer: zope.app.appsetup zope.app.publication zope.app.testing = 3.8.1 zope.browserpage zope.browserresource zope.component zope.container zope.principalregistry zope.ptresource zope.publisher zope.security zope.site zope.traversing And adds these: zope.pagetemplate WebTest zope.app.testing gains this dependency: zope.testbrowser = 4.0.0dev This is a very big change, so I would appreciate anyone who would take a look at these branches before I merge: svn+ssh://svn.zope.org/repos/main/zope.testbrowser/branches/jinty-webtest3 svn+ssh://svn.zope.org/repos/main/zope.app.testing/branches/jinty-testbrowser I plan to merge the branches in the next few weeks if there are no objections. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope] Anonymous security
Yeah -- I rediscovered Proxy roles and that seems like the most straightforward strategy -- not sure if there counter indicators though that would make that strategy problematic. On Sat, Dec 18, 2010 at 10:42 AM, Bart Jansen bart.jan...@esac.climbing.nl wrote: Hi all, When I face a situation like Brian describes I am used to using Proxy roles on the publicly available script to give it permission to do the restricted actions. Is that a good approach or should I not use this? One of the difficulties when using Proxy roles is that they do not propagate to the scripts/methods being called by the script that has the Proxy roles set. Regards, Bart PS. I'm new on the mailing list. My name is Bart Jansen and in my spare time I manage a couple of Zope2 sites for non-profit student sports clubs in the Netherlands. Op 18-12-2010 8:10, Andreas Jung schreef: http://collective-docs.plone.org/security/permissions.html#bypassing-permission-checks (works only from trusted code like browser views or package code - not from PythonScripts) -aj Brian Sullivan wrote: I am looking at a situation (an online self registry process) where I want to allow a user that is not logged in to be able to create a user and do a number of other functions normally reserved for and restricted to logged in users with a fairly elevated rights. I need to perform these functions from a Python script. What is the best strategy for doing this? I am thinking that creating a separate python script that has elevated rights and allowing Anonymous access to it and calling it from a script that does not have elevated rights is the best strategy to manage it. Am I creating a huge security hole by doing this? ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Anonymous security
I am looking at a situation (an online self registry process) where I want to allow a user that is not logged in to be able to create a user and do a number of other functions normally reserved for and restricted to logged in users with a fairly elevated rights. I need to perform these functions from a Python script. What is the best strategy for doing this? I am thinking that creating a separate python script that has elevated rights and allowing Anonymous access to it and calling it from a script that does not have elevated rights is the best strategy to manage it. Am I creating a huge security hole by doing this? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope-dev] zope.testbrowser and WebTest
Hi, I've committed a WebTest integration with testbrowser to a jinty-webtest branch. It basically uses WebTest as a backend to drive a WSGI application. This feature makes it possible to do this in a test: app = make_my_wsgi_app() from webtest import TestApp from zope.testbrowser.wsgi import Browser browser = Browser(TestApp(app)) And then use the browser as normal. I've managed to get the existing tests to run against this browser with two new testing dependencies: WebTest zope.app.wsgi If anyone objects to this feature, or wants to review it before I merge, please let me know. I will definitely fix the documentation and remaining test failure before merging. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest
On Wed, Dec 15, 2010 at 08:53:00AM -0500, Benji York wrote: On Wed, Dec 15, 2010 at 8:06 AM, Brian Sutherland br...@vanguardistas.net wrote: I've committed a WebTest integration with testbrowser to a jinty-webtest branch. It basically uses WebTest as a backend to drive a WSGI application. This sounds like a nice improvement over using wsgi-intercept (http://code.google.com/p/wsgi-intercept/). Exactly. I've been using that for a while, but it finally irritated me too much... I've taken a quick look at the branch. I saw a few of these in the diff: - from zope.testbrowser.testing import Browser +XXX: what to do with this? +XXX from zope.testbrowser.testing import Browser If you can give me some background maybe I can help with these. I now run the tests twice but with a different Browser variable which comes from the test setup. So I need to think of a way of changing the documentation so that it is still clear. The copyright date in src/zope/testbrowser/wsgi.py needs the current year. Sure, I'll fix that. I suspect large chunks of zope.testbrowser.wsgi can be eliminated with judicious refactoring. As you saw, I took the easy way out by copying testing.py. I wanted to see how easy it was to get the tests passing against WebTest. I'll correct my lazyness shortly ;) If you want, I should have time to review your branch again before you merge it. That'd be great. I'll do the refactoring and documentation fixed then ping you again. There's also a single remaining test failure that I may need your help with. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest
On Wed, Dec 15, 2010 at 08:53:00AM -0500, Benji York wrote: If you want, I should have time to review your branch again before you merge it. Ok, I've done some minimal refactoring and am ready to merge the branch when you've reviewed it. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testbrowser and WebTest
On Thu, Dec 16, 2010 at 12:06:36AM +0100, Hanno Schlichting wrote:
On Wed, Dec 15, 2010 at 2:06 PM, Brian Sutherland
br...@vanguardistas.net wrote:
I've managed to get the existing tests to run against this browser with
two new testing dependencies:
WebTest
zope.app.wsgi
zope.app.wsgi shouldn't be a dependency of zope.testbrowser. It's ok
if it's pulled in via a specific extra_requires like a [webtest]
extra, though. Zope2 depends on zope.testbrowser and has no dependency
on any zope.app packages - this must continue to be the case. Required
test dependencies count towards real dependencies here.
I understand your point, but the situation was already pretty nasty as
zope.testbrowser already depended on:
zope.app.appsetup
zope.app.publication
zope.app.testing = 3.8
I agree that any extra dependency is not welcome, but this feature opens
up a path to radically reducing the dependencies of zope.testbrowser.
See below.
The real fix would probably be to move the reusable code out of
zope.app.wsgi into a zope.wsgi package, but that might be more than
you are willing to do now. I do believe some of the Grok people would
be interested in this as well.
There is also another route open now with the addition of the webtest
feature. We could invert the zope.testbrowser - zope.app.testing
dependency.
This is a major re-factoring, but will leave the zope.testbrowser
dependencies looking like this:
install_requires = [
'mechanize=0.2.0',
'setuptools',
'zope.interface',
'zope.schema',
'pytz',
],
extras_require = {
'test': [
'WebOb',
'WebTest',
]
Basically this would require:
* Re-writing the zope.testbrowser.ftests test application as a pure
WSGI app (using WebOb)
* Test only against zope.testbrowser.wsgi and refactor tests to not
use features from zope.app.testing.functional
* Move zope.testbrowser.testing into zope.testbrowser.wsgi and
zope.app.testing.testbrowser
* Leave backwards compatibility imports in place in
zope.testbrowser.testing
I'd be willing to have a look at this if Benji were OK in principle on
this and someone was willing to review it.
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
[Zope] WebDAV lock in Zope 2.11.4
Somehow in testing use of ExternalEditor product I have set the WebDAV lock on a file. The problem is that the lock doesn't seem to appear in the list of locks from the Control Panel, nor does it clear on a restart. I am at loss -- how do I proceed? Is there anyway around this? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] WebDAV lock in Zope 2.11.4
I tried that as well but it didn't appear to find the lock. Then somehow it just went away -- not sure what happened -- it was nothing that I deliberately did as far as I know. Maybe I imagined it. On Thu, Dec 2, 2010 at 1:56 PM, robert rottermann rob...@redcor.ch wrote: Am 02.12.2010 18:53, schrieb Brian Sullivan: Somehow in testing use of ExternalEditor product I have set the WebDAV lock on a file. The problem is that the lock doesn't seem to appear in the list of locks from the Control Panel, nor does it clear on a restart. I am at loss -- how do I proceed? Is there anyway around this? ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) be aware that you must enter a path when scanning for locks. it is not enough just to click on the search buton. i tripped over this once.. robert ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] double opt-in or opt-in with confirmation.
I am looking for a strategy or samples (or even just a hint as to the best process) to create an email opt-in strategy in Zope. I am thinking that what you do is use some sort of hash or unique id creation mechanism (based on the email address, time or something else). Based on what I am seeing -- some sort of MD5(not totally sure what that means but I understand there are Python supplied functions to generate it) hash of a combination of the email address and date/time stored and then sent to the user should guarantee that it is unique, can't be auto generated by some bot and should allow verification on http access using the hash in the url? All the potential solutions I am seeing use Python 2.5 + -- the version of Zope I am using is based on Python 2.4.4. Anybody have any ideas or have implemented something like this? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] double opt-in or opt-in with confirmation.
Looking at the code now -- do you remember where the code to generate the strings is situated? On Tue, Nov 30, 2010 at 12:53 PM, Andreas Jung li...@zopyx.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian Sullivan wrote: I am looking for a strategy or samples (or even just a hint as to the best process) to create an email opt-in strategy in Zope. I am thinking that what you do is use some sort of hash or unique id creation mechanism (based on the email address, time or something else). Based on what I am seeing -- some sort of MD5(not totally sure what that means but I understand there are Python supplied functions to generate it) hash of a combination of the email address and date/time stored and then sent to the user should guarantee that it is unique, can't be auto generated by some bot and should allow verification on http access using the hash in the url? All the potential solutions I am seeing use Python 2.5 + -- the version of Zope I am using is based on Python 2.4.4. Anybody have any ideas or have implemented something like this? As an example look at the Products.EasyNewsletter code. In a recent project we implemented as similar functionality where we generated 32 char longs strings (based on random numbers) and stored the token in an external DB (MongoDB). Sending out a confirmation email and implementing a view for validating the token through an URL is straight forward. - -aj -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQGUBAEBAgAGBQJM9ToyAAoJEADcfz7u4AZjUfcLwIl28L7mHM82x0NmLEizNHKc qcFHi+rQshpg/DqiIgSiYPIgf+608LbbGX5B+ZR4ekBXQBaHJSuzy/kT/xPxlfO2 EaPYnk/ri93PkmGNHFq5ASz5amNn1kwNrOolM32iP24krbzP91vCujFGvIJ4izJk UkacsI6CD8m7q+9StNPjs/uoHEb5ceHBwiC+u/ipBpAnhHNeum0BxNsWIsiQa/Ns M5+ckS0wMfbyQ+FYEtA7+KnmSiDTnHmg0M0Ii3INXbEZgx9h7u+ezHDFpxU5V2UD kYBf+3eTFu5rRmqbiHMNwnEzBHt1pe8QHPns3+3v15fi7b3i3r4x7PxHHDLbawQq +wtLDlCBzz02vcKjHd0aItSTCxCPu3vOPgCY/hMpsCnz2ElQIgBb6Np0tl6Vt1D6 A3XIkM39X1+wOQUgiqWgylvKvYosx9rrUM+zJ3L6zNZt6YpaDT8+d3cA3hqBNMUA fqRaDaMFL7xclCzB4BOUoEUYwHdjzC0= =uy5J -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Clock server
I am about to put the clock server function of Zope to work and am thinking about what happens if the server fires its request while a request is already in progress. Is there some sort of built in lock preventing this from happening? If not what is the most straightforward failsafe way to implement something that would lock on entry and unlock on exit? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope directory in the Cloud
Is it possible to use some sort of Cloud service as a directory in Zope 2 (I guess what I am asking for is some sort of product to do this)? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope directory in the Cloud
Brian Sullivan wrote: Is it possible to use some sort of Cloud service as a directory in Zope 2 (I guess what I am asking for is some sort of product to do this)? Directory? Cloud as storage? Essentially yes -- I would like to transfer the heavy lifting -- delivery of content that will probably include video and other large files to an external server while keeping the management of access and tracking use apparently local (and simple hopefully -- using what I already have). I would like it to appear as a directory in Zope to avoid cross domain scripting issues. ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] SVN: zope.interface/branches/jinty-mem/src/zope/interface/interface.py Improve CPU performance of previous memory optimization
On Tue, Nov 09, 2010 at 03:01:09PM -0500, Tres Seaver wrote:
I think the is a possible threading issue with Element.setTaggedValue
and Specification.subscribe - if two threads called the method
concurrently, then one of the values might be lost. I think the
correct way to do it would be:
tv = self.__tagged_values
if tv is None:
tv = self.__dict__.setdefault('_Element__tagged_values', {})
tv[tag] = value
This does bring the name mangling back though.
Thanks, I fixed the threading issue in Specification.subscribe. Given
that that part of subscribe is not run very often, I think we can live
with limited name mangling.
I'm pretty sure we can safely neglect threading issues here: no sane
code will call 'setTaggedValue' except at import time, when we should be
serialized by Python's own import lock.
Great, I quoted you on that ;)
The setdefault fix for the threading issue is not compatible with the
use of __slots__. I couldn't find another way to do it.
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.interface memory optimization
On Wed, Nov 10, 2010 at 12:07:00PM +0100, Charlie Clark wrote: Am 10.11.2010, 01:40 Uhr, schrieb Brian Sutherland br...@vanguardistas.net: On the micro benchmarks, the only difference I see is a memory improvement which I think will not survive in practice because most ... ? Er, somehow my thoughts didn't make it into text: attributes are zope.schema fields (i.e. subclasses of Attribute). To really make that work, you'd have to add __slots__ to most of the classes in zope.schema as well. It might be purely academic but if this does significantly reduce memory use then it might be worth pursuing. Turns out it's not that academic. I tried a bit harder and managed to apply __slots__ to the Element, Attribute and Method classes in zope.interface. That saved an additional 0.5 % of memory (bytes). I have a small patch to zope.schema that fixes the breakage there. It looks like there is an additional 0.5% available if zope.schema is refactored to use slots as well. Slots are supposed to be beneficial when you've got lots of objects around. Zope schema fields are pretty static anyway so I don't see a great penalty in having to add slots to them, although obviously a migration of a larger project would probably be a pain. Searching for __dict__ here: http://zope3.pov.lt/trac/browser/zope.schema/trunk/src/zope/schema/_bootstrapfields.py Will give you a good idea of the problems. Field.bind and ValidatedProperty both depend heavily on __dict__. For ValidatedProperty in particular I have no idea how that would work when using slots. Back to the micro-optimisation. At Jeff Rush's talk at PyCon this year he covered the penalties incurred by . lookups so that might be something that could be investigated but might it also be possible to memoize the method calls? I keep forgetting when you can't do that. I had a look at http://en.wikipedia.org/wiki/Memoization, but I don't see how it could be applied in this case. Charlie -- Charlie Clark Managing Director Clark Consulting Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope] Date utility for Zope (V2)
I have recurring events that I want to schedule (the events are sending email, Twitter DMs and Facebook messages -- at least those are the possible ones that I have identified so far). I need some flexible way to both specify the intervals and interpret the intervals so that the next event in a sequence can be specified in real time terms and scheduled. The event schedule at least in plain English could be something like every weekday at 8:45, every Monday at 9:00 and 1:00, every day at 2:00, the first of every month at 9:00, every other Monday. I realize that I may have to settle for something that is not quite as flexible as I imagine and at least for now I am just trying to see if someone else has solved this problem or one similar and has something I can use as a base. I do have a way to trigger and execute the events at specific times. On Wed, Nov 10, 2010 at 5:00 AM, Lennart Regebro rege...@gmail.com wrote: On Wed, Nov 10, 2010 at 02:00, Brian Sullivan briansulli...@gmail.com wrote: I guess I having a mental block then. To use it from what I know I would have create a Zope 2 product from it somehow or create callable scripts in the ZMI from the provided code. Neither seems reasonable. Is there other way to use the code that I am not understanding? Well, you need to do something with the rrules, yes. And that obviously needs to be done in some sort of Zope product. And you will need to write that, so it does what you want it to do. But the dtutil module itself you simply use from Python like this: from dtutil import rrule rrule.whatevereblahlahlah() I have the feeling (but I could be wrong) that there is a very long distance from what you want to what you say you want. Maybe you can explain what you actually want to do, your usecases etc. -- Lennart Regebro, Colliberty: http://www.colliberty.com/ Telephone: +48 691 268 328 ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Date utility for Zope (V2)
I had a look at tutorials to create products for Zope2 and at first glance they are beyond by pay grade. Maybe I will look at them more closely to see if I might have been hasty but likely it will be easier for me and take less time just to write what I need in script and forgo the I would like to do stuff. Regardless thanks for the help. On Wed, Nov 10, 2010 at 5:30 PM, Lennart Regebro rege...@gmail.com wrote: On Wed, Nov 10, 2010 at 15:25, Brian Sullivan briansulli...@gmail.com wrote: I have recurring events that I want to schedule (the events are sending email, Twitter DMs and Facebook messages -- at least those are the possible ones that I have identified so far). Right, so you need to make these events as you normally write your applications. I need some flexible way to both specify the intervals and interpret the intervals so that the next event in a sequence can be specified in real time terms and scheduled. And that's what dateutil.rrule is for. -- Lennart Regebro, Colliberty: http://www.colliberty.com/ Telephone: +48 691 268 328 ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] zope.interface memory optimization
On Tue, Nov 09, 2010 at 01:21:00AM +0200, Marius Gedminas wrote:
On Mon, Nov 08, 2010 at 03:35:09PM +0100, Brian Sutherland wrote:
I've committed 2 patches to a jinty-mem branch of zope.interface.
Together these patches reduce the startup memory use of my ZTK based
application by 3%. Is that enough to justify their risk/complexity?
https://mail.zope.org/pipermail/checkins/2010-November/052516.html
https://mail.zope.org/pipermail/checkins/2010-November/052517.html
If no-one replies, I'll assume that 3% is just not enough to be
interesting and do nothing;)
It's so interesting I'm going to ask for even more numbers: how much is
that 3% in kilobytes, or objects?
Ok, you asked for it ;)
The 3% was in kilobytes as measured using /bin/ps on OSX leopard with
python2.6 from MacPorts. The app I measured is a ZTK 1.0 based web
application.
My application without the changes:
Objects:245614
RSS memory (kb):59116
My application with the changes:
Objects:227147
RSS memory (kb):57180
I also did a micro-benchmark (more below) which indicate that when
creating an interface with 5 attributes and 5 methods you save 41% in
kilobytes and 34% in objects.
Is there any measurable speed impact
for application startup or test runs (faster/slower)?
My app doesn't startup measurably faster or slower. The ZTK tests seem
to run a little faster, but the error is high on my laptop:
Without patches:
./bin/test-ztk 71.93s
./bin/test-ztk 70.49s
./bin/test-ztk 70.31s
With patches:
./bin/test-ztk 70.28s
./bin/test-ztk 70.00s
./bin/test-ztk 69.98s
I also did some micro-benchmarking (script is attached). There were
repeatable results which indicate that startup will be faster and
runtime only probably faster:
Without patches:
Performance bench
-
one interface, 5 methods, 5 functions: 181.00 usec/pass
five inheriting interfaces: 355.76 usec/pass
one interface: 59.79 usec/pass
query uninitialized tagged value: 1.26 usec/pass
query initialized tagged value: 1.27 usec/pass
Memory bench
Memory before (kb):
RSS VSZ
33684 109780
objects created: 150001
Memory after (kb):
RSS VSZ
47856 124116
With patches:
Performance bench
-
one interface, 5 methods, 5 functions: 165.97 usec/pass
five inheriting interfaces: 341.22 usec/pass
one interface: 53.03 usec/pass
query uninitialized tagged value: 0.75 usec/pass
query initialized tagged value: 1.37 usec/pass
Memory bench
Memory before (kb):
RSS VSZ
33328 109524
objects created: 99001
Memory after (kb):
RSS VSZ
41728 117972
Otherwise I'm +1, assuming this doesn't somehow make the code 10%
slower.
Marius Gedminas
--
http://pov.lt/ -- Zope 3/BlueBream consulting and development
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
--
Brian Sutherland
#
# Perdormance Benchmark
#
import os
import timeit
import subprocess
from zope.interface import Interface, Attribute
print Performance bench
print -
s1 = \
class I1(Interface):
a1 = Attribute('one')
a2 = Attribute('two')
a3 = Attribute('thee')
a4 = Attribute('four')
a5 = Attribute('five')
def f1(arg):
pass
def f2(arg):
pass
def f3(arg):
pass
def f4(arg):
pass
def f5(arg):
pass
t = timeit.Timer(stmt=s1, setup=from zope.interface import Interface,
Attribute)
print one interface, 5 methods, 5 functions: %.2f usec/pass % (100 *
t.timeit(number=1)/1)
s1 = \
class I1(Interface):
pass
class I2(I1):
pass
class I3(I2):
pass
class I4(I3):
pass
class I5(I4):
pass
t = timeit.Timer(stmt=s1, setup=from zope.interface import Interface,
Attribute)
print five inheriting interfaces: %.2f usec/pass % (100 *
t.timeit(number=1)/1)
s1 = \
class I1(Interface):
pass
t = timeit.Timer(stmt=s1, setup=from zope.interface import Interface,
Attribute)
print one interface: %.2f usec/pass % (100 * t.timeit(number=1)/1)
setup = \
from zope.interface import Interface, Attribute
class I1(Interface):
pass
s1 = \
I1.queryTaggedValue('bob')
t = timeit.Timer(stmt=s1, setup=setup)
print query uninitialized tagged value: %.2f usec/pass % (100 *
t.timeit(number=5)/5)
setup = \
from zope.interface import Interface, Attribute
class I1(Interface):
pass
I1.setTaggedValue('bob', 'bobby')
s1 = \
I1.queryTaggedValue('bob')
t = timeit.Timer(stmt=s1, setup=setup)
print
Re: [Zope-dev] zope.interface memory optimization
On Tue, Nov 09, 2010 at 03:49:38PM +0100, Charlie Clark wrote:
Am 08.11.2010, 15:35 Uhr, schrieb Brian Sutherland
br...@vanguardistas.net:
If no-one replies, I'll assume that 3% is just not enough to be
interesting and do nothing;)
Hi Brian,
thanks for sharing this but it looks to me like a micro-optimisation that
isn't really worth going against best practice for - by using explicit
name-mangling.
-return self.__tagged_values.get(tag, default)
+return getattr(self, '_Element__tagged_values', {}).get(tag,
default)
In response to Marius' comments I changed the patch a bit earlier to
this:
def queryTaggedValue(self, tag, default=None):
Returns the value associated with 'tag'.
tv = self.__tagged_values
if tv is None:
return default
return tv.get(tag, default)
Which is is faster as well as not doing the explicit name mangling you
object to.
Any improvements higher up the stack (using slots, different Python
version or compilers like LLVM) are likely to have a more significant
effect.
That would be great, when it arrives.
Actually, in my case, memory is the issue. So the only optimization
that'll really help in a big way is if multiple python processes running
similar code start sharing data.
Some people have poked at that problem, but I havn't seen anything
significant yet.
Might be worth timing a slots-based implementation.
I just tried to add slots to Element, Attribute and Method classes, but
it there's too many tests that break to benchmark it running the ZTK
tests.
On the micro benchmarks, the only difference I see is a memory
improvement which I think will not survive in practice because most
Charlie
--
Charlie Clark
Managing Director
Clark Consulting Research
German Office
Helmholtzstr. 20
Düsseldorf
D- 40215
Tel: +49-211-600-3657
Mobile: +49-178-782-6226
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
--
Brian Sutherland
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.interface memory optimization
On Wed, Nov 10, 2010 at 12:14:54AM +0200, Marius Gedminas wrote: On Tue, Nov 09, 2010 at 05:04:11PM +0100, Brian Sutherland wrote: Might be worth timing a slots-based implementation. I just tried to add slots to Element, Attribute and Method classes, but it there's too many tests that break to benchmark it running the ZTK tests. On the micro benchmarks, the only difference I see is a memory improvement which I think will not survive in practice because most ... ? Er, somehow my thoughts didn't make it into text: attributes are zope.schema fields (i.e. subclasses of Attribute). To really make that work, you'd have to add __slots__ to most of the classes in zope.schema as well. Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope] Date utility for Zope (V2)
I am looking for a standard way (from a python script in Zope) to define and manipulate time intervals for periodic events. I need to define things like every weekday at 8 am, the 2nd day of every month at 8am, every weekday at 8am, 10am and 3pm in addition to easy things like everyday at 1pm. Does such a standard exist - perhaps with a library to manage the conversion to real times? On another forum this http://labix.org/python-dateutil was suggested -- it looks to be close to what I was thinking -- but I am not totally sure how I can use it in Zope (a level of incompetence issue no doubt). Is there something else Zope(not Plone) ready? Or alternately how would I use this in Zope? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Date utility for Zope (V2)
I guess I having a mental block then. To use it from what I know I would have create a Zope 2 product from it somehow or create callable scripts in the ZMI from the provided code. Neither seems reasonable. Is there other way to use the code that I am not understanding? On Tue, Nov 9, 2010 at 6:11 PM, Lennart Regebro rege...@gmail.com wrote: On Tue, Nov 9, 2010 at 22:12, Brian Sullivan briansulli...@gmail.com wrote: On another forum this http://labix.org/python-dateutil was suggested -- it looks to be close to what I was thinking -- I second that. but I am not totally sure how I can use it in Zope (a level of incompetence issue no doubt). You would use it as you would use it outside Zope. No difference, it's just all Python. //Lennart ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope-dev] zope.interface memory optimization
I've committed 2 patches to a jinty-mem branch of zope.interface. Together these patches reduce the startup memory use of my ZTK based application by 3%. Is that enough to justify their risk/complexity? https://mail.zope.org/pipermail/checkins/2010-November/052516.html https://mail.zope.org/pipermail/checkins/2010-November/052517.html If no-one replies, I'll assume that 3% is just not enough to be interesting and do nothing;) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope] Persist password in CookieCrumbler
Thanks -- will have a look. On Fri, Oct 22, 2010 at 3:43 AM, Peter Bengtsson pe...@fry-it.com wrote: I wrote something a long time ago which did this. Download http://www.issuetrackerproduct.com/Download#CookieCrumblerIssueTrackerProduct And read some of the source I think what you have to do is override its setAuthCookie method somehow and there you can set 'expires' to be a date far in the future. On 21 October 2010 23:28, Brian Sullivan briansulli...@gmail.com wrote: Can I persist the password using CookieCrumbler (in addition to the user name)? Has anybody made this modification and can supply the modified product or code. I made a stab at it but obviously my level of understanding is not up to snuff 'cause I can't get it to work. What are the implications/problems that might result from doing this? ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Persist password in CookieCrumbler
On Fri, Oct 22, 2010 at 12:34 PM, Tres Seaver tsea...@palladion.com wrote: The obvious issue with a beyond-this-session auth cookie is that it enables anybody who can run that browser / profile to authenticate as the user being persisted. I would consider this an unacceptable risk for any site where the authentication was intended for anything more than keep spambots out (i.e., you might as well be using OpenID). Isn't this about the same risk as the browser saving the id/password pair for the site? Certainly on a public or multiuser machine this would not be a good idea and appropriate warnings should be given. (it seems to me that all browsers do this and most users take advantage of this) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Persist password in CookieCrumbler
Can I persist the password using CookieCrumbler (in addition to the user name)? Has anybody made this modification and can supply the modified product or code. I made a stab at it but obviously my level of understanding is not up to snuff 'cause I can't get it to work. What are the implications/problems that might result from doing this? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Asynchronous triggering of events
So I installed 2.11.4 -- the latest that I could find and still the same issue. I get the impression from the age of the Zope installs, the state of disrepair and link rot of the zope.org site that Zope is in its final death throes as an independent entity ? I tried to look for an independent version of Plone that I can install on an existing Zope-- the only install I could find is a complete installer including Zope. Is Plone the new Zope? On Mon, Oct 18, 2010 at 5:27 PM, Chris Withers ch...@simplistix.co.uk wrote: On 18/10/2010 22:25, Brian Sullivan wrote: zopetcl -- is that a unix program? I am using Windows unfortunately. It might well be on your version of Zope, it works fine in later versions, though... That said, if you take a look at Stepper, you'll see that I documented how to get it all to work on Windows (use Scheduled Tasks to drive it) cheers, Chris PS: But seriously, how much of a masochist are you? Zope 2.9 on Windows?! ;-) -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Plone the new Zope?
I haven't done any Zope work for quite a while and have just restarted out of necessity. Things seem to have changed in the last few years. I get the impression from the age of the Zope installs, the state of disrepair and link rot of the zope.org site that Zope is in its final death throes as an independent entity ? I tried to look for an independent version of Plone that I can install on an existing Zope = the only install I could find is a complete installer including Zope. Is Plone the new Zope? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Asynchronous triggering of events
I am back working with Zope 2.9.1 (yes I know it is long in the tooth but I am trying to avoid adding the pain of upgrading, having products fail and all the usually comes with upgrading if I can). I am looking for a strategy to schedule and carry out tasks at periodic future dates. I had a look at Xron (http://www.zope.org/Members/lstaffor/Xron/) but it either doesn't seem to work with 2.9.1 (or I can't get it to work). I install it in the products directory but it doesn't show up in the product directory when Zope restarts and naturally none of the objects it is supposed to supply can be added. I haven't investigated what the problem is (and am not even totally sure how to go about it). Is there something isn't obvious that I might be missing? Barring getting it to work are there any other alternatives? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Asynchronous triggering of events
Clockserver seems to install -- can't find any docs on how to use it though. All the links for info at the download site appear to be dead. Any idea where docs might be located? On Mon, Oct 18, 2010 at 4:18 PM, Jeff Peterson jeff.peter...@crary.com wrote: I'd start by looking at the logs to see what's failing, should be a traceback there somewhere. Other than that there is clockserver, but I am not sure that was/is available in the version you are using. -- Jeffrey D Peterson Webmaster Crary Industries, Inc. 237 12th St NW West Fargo, ND 58078 P: 701-499-5928 E: jeff.peter...@crary.com -Original Message- From: zope-boun...@zope.org [mailto:zope-boun...@zope.org] On Behalf Of Brian Sullivan Sent: Monday, October 18, 2010 2:51 PM To: Zope Users Subject: [Zope] Asynchronous triggering of events I am back working with Zope 2.9.1 (yes I know it is long in the tooth but I am trying to avoid adding the pain of upgrading, having products fail and all the usually comes with upgrading if I can). I am looking for a strategy to schedule and carry out tasks at periodic future dates. I had a look at Xron (http://www.zope.org/Members/lstaffor/Xron/) but it either doesn't seem to work with 2.9.1 (or I can't get it to work). I install it in the products directory but it doesn't show up in the product directory when Zope restarts and naturally none of the objects it is supposed to supply can be added. I haven't investigated what the problem is (and am not even totally sure how to go about it). Is there something isn't obvious that I might be missing? Barring getting it to work are there any other alternatives? ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Asynchronous triggering of events
Clockserver seems to need an external C module that is only available for Xnix - I am unfortunately on Windows. 2010-10-18T16:27:01 ERROR Zope Couldn't install Xron Traceback (most recent call last): File C:\Program Files\Zope-2.9.1\lib\python\OFS\Application.py, line 763, in install_product global_dict, global_dict, silly) File d:\Zope-Instance2.9\Products\Xron\__init__.py, line 40, in ? import Dispatcher File d:\Zope-Instance2.9\Products\Xron\Dispatcher.py, line 42, in ? import ZODB, ZODB.ZApplication ImportError: No module named ZApplication is the error I get on startup related to Xron. Not sure what ZApplication is or where it is supposed to be. On Mon, Oct 18, 2010 at 4:46 PM, Brian Sullivan briansulli...@gmail.com wrote: Clockserver seems to install -- can't find any docs on how to use it though. All the links for info at the download site appear to be dead. Any idea where docs might be located? On Mon, Oct 18, 2010 at 4:18 PM, Jeff Peterson jeff.peter...@crary.com wrote: I'd start by looking at the logs to see what's failing, should be a traceback there somewhere. Other than that there is clockserver, but I am not sure that was/is available in the version you are using. -- Jeffrey D Peterson Webmaster Crary Industries, Inc. 237 12th St NW West Fargo, ND 58078 P: 701-499-5928 E: jeff.peter...@crary.com -Original Message- From: zope-boun...@zope.org [mailto:zope-boun...@zope.org] On Behalf Of Brian Sullivan Sent: Monday, October 18, 2010 2:51 PM To: Zope Users Subject: [Zope] Asynchronous triggering of events I am back working with Zope 2.9.1 (yes I know it is long in the tooth but I am trying to avoid adding the pain of upgrading, having products fail and all the usually comes with upgrading if I can). I am looking for a strategy to schedule and carry out tasks at periodic future dates. I had a look at Xron (http://www.zope.org/Members/lstaffor/Xron/) but it either doesn't seem to work with 2.9.1 (or I can't get it to work). I install it in the products directory but it doesn't show up in the product directory when Zope restarts and naturally none of the objects it is supposed to supply can be added. I haven't investigated what the problem is (and am not even totally sure how to go about it). Is there something isn't obvious that I might be missing? Barring getting it to work are there any other alternatives? ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Asynchronous triggering of events
zopetcl -- is that a unix program? I am using Windows unfortunately. On Mon, Oct 18, 2010 at 5:03 PM, Chris Withers ch...@simplistix.co.uk wrote: On 18/10/2010 20:51, Brian Sullivan wrote: I am back working with Zope 2.9.1 (yes I know it is long in the tooth but I am trying to avoid adding the pain of upgrading, having products fail and all the usually comes with upgrading if I can). I I would just use zopectl run and a script of your choice triggered by cron. Stepper provides a nice set of utils for working in this pattern... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] zope.app.appsetup and RootErrorReportingUtility
On Mon, Aug 30, 2010 at 03:01:24PM +0200, Adam GROSZER wrote: Hello, zope.app.appsetup installs the following RootErrorReportingUtility: ensureUtility(root_folder, IErrorReportingUtility, '', RootErrorReportingUtility, copy_to_zlog=False) My problem with this is that copy_to_zlog=False, that means if your app is *really* hosed you don't get the error logged to the usual error.log or event.log and are also *unable* to access the UI of the utility. Anyone objects a change to copy_to_zlog=True as default? +1 -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] docs.zope.org automation
On Mon, Aug 02, 2010 at 06:34:21PM +0200, Jens Vagelpohl wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 8/2/10 18:26 , Martijn Faassen wrote: http://packages.python.org/distribute/pkg_resources.html You can probably create a Distribution object somehow (handwave) from a path (whether that's the path of the package or the path the package is in, not sure). The issue, just like with pkginfo.Develop, is that I can't find any function or method that finds package information in subfolders. If you look at... http://packages.python.org/distribute/pkg_resources.html#getting-or-creating-distributions ...there's this function find_distributions which supposedly offers subfolder searching, but it just doesn't work. At least it doesn't do what I would expect when I read the documentation. It finds nothing. Do you have access to an source distribution (tarball or zip) for the package involved? In the source distribution the PKG-INFO file is always at the top level. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxW840ACgkQRAx5nvEhZLK0MgCfWNsSL47dimbU9TlIZNSRq07h HLIAoIYYLdYUsA5PhYrtqhu9EaVk8Nip =GbbC -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope] Zope for .NET
On Mon, Apr 19, 2010 at 2:04 AM, Chris Withers ch...@simplistix.co.uk wrote: Brian Sullivan wrote: Yes -- would love to use it here too but unfortunately it doesn't fit the scenario. What about it doesn't fit? It doesn't fit because the customer already has a system using .NET that has to tightly integrate and has said they want a .NET system. ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope for .NET
On Mon, Apr 19, 2010 at 10:07 AM, Jim Pharis binbr...@gmail.com wrote: Maybe try DotNetNuke. I have no in depth knowledge of it but I believe it is designed using component architecture and its open source. Yes DotNetNuke was one that I found and it seems like the most likely candidate -- I was just wondering if there were other options. ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope for .NET
On Mon, Apr 19, 2010 at 11:00 AM, Chris Withers ch...@simplistix.co.uk wrote: Brian Sullivan wrote: It doesn't fit because the customer already has a system using .NET that has to tightly integrate and has said they want a .NET system. And IronPython isn't .NET how exactly?! IronPython isn't Zope though is it ?-- which is what I was referring to and where there is some potential leverage. Using Python doesn't really help in this situation (other than the fact that I am more familiar with Python than C)-- the existing system is done with C# --and they want to be able to manage, develop and maintain using the same resources. Is there something about IronPython that I might be missing? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope for .NET
I am looking for an open source .NET framework that includes user management, authentication, multiple configurable levels of security, tools for html generation and is data base agnostic or includes some sort of built in data base. It would be nice if it had a host of open source add ins. Basically Zope functionality for .NET. Is there anything like that out there -- or even something close? ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope for .NET
Tried Zope? d8) Yes -- would love to use it here too but unfortunately it doesn't fit the scenario. But thanks for the suggestion will keep it mind for future projects. ;-) ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope Performance with Multiple Mount Points
We've recently moved to a Zope configuration with approximately 30 separate databases mounted at mount points within a main database. Previously, we ran a single database which was approaching 300GB in size. We chose to split the database to reduce Data.fs size. We now run 30 separate ZEO Servers (on a single machine). However, after switching to this configuration we've noticed some performance issues. Initially, after a client is restarted and builds out it's caches the site runs very quickly. However as the memory usage grows the performance degrades. If the client gets into swap the client practically dies. We had 10,000 object caches on each of the databases with 2 threads on the client. 30 x 10,000 x 2 = 600,000 objects in memory. We've since reduced the cache sizes to 1,000 or 30 x 1,000 x 2 = 60,000 objects. This seemed to extend the life of a client between restarts. Generally, our client machines will hover around 97%-99% memory usage and 90%-100% CPU (on a 2 CPU machine). We still experience periodic performance problems and are looking for any input that might help us address them. Thanks for your input, -- Brian Brinegar Web Services Coordinator Engineering Computer Network ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Performance with Multiple Mount Points
Our goal with this configuration was to limit the ability of a single site to lock the entire database during a large transaction. Are ZEO Server transaction locks one per storage or one per server? If the locks are per storage rather than per server I believe we could accomplish our goal with less servers, in which case is there an upper limit on the number of storages a single server can reasonably handle? Thanks, -Brian Andreas Jung wrote: Brian Brinegar wrote: We've recently moved to a Zope configuration with approximately 30 separate databases mounted at mount points within a main database. Previously, we ran a single database which was approaching 300GB in size. We chose to split the database to reduce Data.fs size. We now run 30 separate ZEO Servers (on a single machine). However, after switching to this configuration we've noticed some performance issues. Initially, after a client is restarted and builds out it's caches the site runs very quickly. However as the memory usage grows the performance degrades. If the client gets into swap the client practically dies. We had 10,000 object caches on each of the databases with 2 threads on the client. 30 x 10,000 x 2 = 600,000 objects in memory. We've since reduced the cache sizes to 1,000 or 30 x 1,000 x 2 = 60,000 objects. This seemed to extend the life of a client between restarts. No idea what's running onside your system and inside your Zope - 1000 objects seems way to small for almost any case. Also 30 ZEO servers seems kind of extreme. We are running similar a big applications and we are running with five or six ZEO servers - not 30. -aj Generally, our client machines will hover around 97%-99% memory usage and 90%-100% CPU (on a 2 CPU machine). We still experience periodic performance problems and are looking for any input that might help us address them. Thanks for your input, -- Brian Brinegar Web Services Coordinator Engineering Computer Network ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Dieter Mauer's Reference Product
Our university relies heavily on a Zope product based on Dieter Maurer's Reference product. Recently, we upgraded from Zope 2.9.6 to Zope 2.11.x and found some changes in behavior. In short the Reference product creates a Symlink like pointer in the Zope hierarchy. Dieter's product can be found on his site at: http://www.dieter.handshake.de/pyprojects/zope/index.html#bct_sec_5.9 First, the security machinery now prevents access to attributes of References through page template path notation. For example, the following fails: tal:content=container/MyReference/property_name Traceback: ... * Module zope.tales.expressions, line 217, in __call__ * Module Products.PageTemplates.Expressions, line 133, in _eval * Module zope.tales.expressions, line 124, in _eval * Module Products.PageTemplates.Expressions, line 82, in boboAwareZopeTraverse * Module OFS.Traversable, line 301, in restrictedTraverse * Module OFS.Traversable, line 232, in unrestrictedTraverse __traceback_info__: ([], 'property_name') Unauthorized: You are not allowed to access 'property_name' in this context Interestingly, the same access via dot notation works: tal:content=python:container.MyReference.property_name There were substantial changes to Traversable.py between versions which seem to cause the problem. Any suggestion on how to fix this would be greatly appreciated. Second, through path notation or URL traversal, References under the previous version of Zope would default to using methods / objects within the target before falling back to acquisition. Under Zope 2.11 acquired methods/objects take priority (only when traversed). For example, assuming there is an index_html in the root as well as in the target, and using the following code: tal:content=container/MyReference/index_html/absolute_url_path Zope 2.11 yields the path to the acquired index_html: /index_html Zope 2.9.6 yields the path to the index_html in the target: /Path/To/Target/index_html Again, through python, both yield the second, desired output. I realize this is an obscure product, and the changes seem to have to do with the Five implementation. At this point we are looking for options to restore the desired functionality which does not require perpetually running Zope 2.9.x One option may be to change everything to dot notation, however I would at least like to understand why this change occurred. Thanks for reading, -- Brian Brinegar Web Services Coordinator Engineering Computer Network ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Dieter Mauer's Reference Product
Dieter, You've just made my week! I'm glad that my failure to understand how all of this works has shed some light on the problem. Thank you, Brian Dieter Maurer wrote: Dieter Maurer wrote at 2010-3-16 17:42 +0100: Brian Brinegar wrote at 2010-3-16 10:12 -0400: Our university relies heavily on a Zope product based on Dieter Maurer's Reference product. Recently, we upgraded from Zope 2.9.6 to Zope 2.11.x and found some changes in behavior. In short the Reference product creates a Symlink like pointer in the Zope hierarchy. Dieter's product can be found on his site at: http://www.dieter.handshake.de/pyprojects/zope/index.html#bct_sec_5.9 First, the security machinery now prevents access to attributes of References through page template path notation. For example, the following fails: tal:content=container/MyReference/property_name Traceback: ... * Module zope.tales.expressions, line 217, in __call__ * Module Products.PageTemplates.Expressions, line 133, in _eval * Module zope.tales.expressions, line 124, in _eval * Module Products.PageTemplates.Expressions, line 82, in boboAwareZopeTraverse * Module OFS.Traversable, line 301, in restrictedTraverse * Module OFS.Traversable, line 232, in unrestrictedTraverse __traceback_info__: ([], 'property_name') Unauthorized: You are not allowed to access 'property_name' in this context This is a bug/weakness in Zope which affects the traversal methods (used for TALES path expressions): When a value is retrieved during traversal via __bobo_traverse__ which does not have its own security declarations (impossible for a simple datatype), then the traversal insists that it is the same object (verified by object identity) than the object retrieved via getattr (guarded_getattr, to be precise). This drastically restricts the access to simple values via traversal if __bobo_traverse__ is defined. Reference grew a __bobo_traverse__ method to work around a (apparent) Five bug as delivered with Zope 2.9. Maybe, the __bobo_traverse__ method is not longer necessary for Zope 2.11. Try to comment it out. ... Second, through path notation or URL traversal, References under the previous version of Zope would default to using methods / objects within the target before falling back to acquisition. Under Zope 2.11 acquired methods/objects take priority (only when traversed). For example, assuming there is an index_html in the root as well as in the target, and using the following code: tal:content=container/MyReference/index_html/absolute_url_path Zope 2.11 yields the path to the acquired index_html: /index_html Zope 2.9.6 yields the path to the index_html in the target: /Path/To/Target/index_html Again, through python, both yield the second, desired output. This sounds strange -- almost unbelievable. I will look into it within the next few days and report back. Thanks to your problem report, I have much better understood the problem reported by J Cameron Cooper for Zope 2.9. The problem has not been a Five problem. Instead, it was caused by a confusion whether the traversal methods should be resolved with respect to the reference or its target. The primary implementation resolved them with respect to the reference and then could not traverse with respect to the target -- J Cameron's problem. The __bobo_traverse__ method partially fixed this again using an explicit proxy (which takes into account both reference and target) but triggered the security weakness in Zope's traversal for simple values. A bug in its implementation (a missing aq_base(...)) caused the wrong acquisition context. After the improved understanding, I can handle traversal methods without a need for __bobo_traverse__. This fixes both of the problems you have observed. I will write some tests and then publish References as Products.References on PyPI in the next days. Thank you for your problem report! -- Dieter -- Brian Brinegar Web Services Coordinator Engineering Computer Network ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] Uses of the ZTK and how it relates to management
On Thu, Mar 04, 2010 at 10:09:26AM -0500, Jim Fulton wrote: On Thu, Mar 4, 2010 at 5:35 AM, Hanno Schlichting ha...@hannosch.eu wrote: On Thu, Mar 4, 2010 at 7:19 AM, Christian Theune c...@gocept.com wrote: A thought that came up when reading this paragraph: another option restructuring/grouping to reduce the amount of packages may be to join smaller packages with weird boundaries into larger ones again. (Not that I suggest this to be an ultimate option, nor do I know from the top of my head any candidates for this, but we can keep that on the list of options we have.) I think this is a good idea, but I wouldn't want to do it on a package level. Rather do it on the distribution level. Once the distutils2 improvements are available, we have the means to say distribution A obsoletes B. As a simple example that would allow us to put zope.event into the zope.component distribution, without having to change any import paths or setup.py install_requires lines. The zope.component distribution would have the metadata to say I obsolete zope.event, so if someone has such a version of zope.component, requirements of the zope.event distribution would be automatically satisfied. This same method could be taken to build more functional distribution out of related packages we have today. These distributions might also be easier to market, document and explain. But they come with the downside of more buy-in per distribution. Figuring out how packages are actually used and which ones are used independently is no small task. Your description of the mechanism sounds interesting. In the specific case of zope.event, I'd prefer it stay separate. I want developers to be able to publish events without having to commit to a subscription mechanism. For example, ZODB depends on zope.event so it can generate events and provide a generic hook mechanism. I don't want it to depend on zope.component. I just committed a jinty-optional-event branch for zope.component that's an experiment as to how to make the dependency on zope.event optional. zope.event will still be used if it is around but zope.component will provide it's own implementation if not. Other packages which only depend on zope.component can break their dependency on zope.event. Ideally, I'd like other projects to adopt zope.event, or for something like zope.event to be included in the standard library, although, I'm unlikely to push that politically, so it will probably never happen. :/ Jim -- Jim Fulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Uses of the ZTK and how it relates to management
On Thu, Mar 04, 2010 at 11:19:52AM -0500, Benji York wrote: On Thu, Mar 4, 2010 at 11:09 AM, Brian Sutherland br...@vanguardistas.net wrote: On Thu, Mar 04, 2010 at 10:09:26AM -0500, Jim Fulton wrote: In the specific case of zope.event, I'd prefer it stay separate. I want developers to be able to publish events without having to commit to a subscription mechanism. For example, ZODB depends on zope.event so it can generate events and provide a generic hook mechanism. I don't want it to depend on zope.component. I just committed a jinty-optional-event branch for zope.component that's an experiment as to how to make the dependency on zope.event optional. Maybe I'm missing something, but zope.event is so minimal I can't see that making optional is worth the effort. One point is backwards compatibility, the other is to allow zope.component subscribers to listen to ZODB events if the ZODB only depends on zope.event but not on zope.component. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Uses of the ZTK and how it relates to management
On Thu, Mar 04, 2010 at 11:19:52AM -0500, Benji York wrote: On Thu, Mar 4, 2010 at 11:09 AM, Brian Sutherland br...@vanguardistas.net wrote: On Thu, Mar 04, 2010 at 10:09:26AM -0500, Jim Fulton wrote: In the specific case of zope.event, I'd prefer it stay separate. I want developers to be able to publish events without having to commit to a subscription mechanism. For example, ZODB depends on zope.event so it can generate events and provide a generic hook mechanism. I don't want it to depend on zope.component. I just committed a jinty-optional-event branch for zope.component that's an experiment as to how to make the dependency on zope.event optional. Maybe I'm missing something, but zope.event is so minimal I can't see that making optional is worth the effort. Actually, I misunderstood your question. Personally, I also don't think it's worth the effort. But it's come up a few times already, so is obviously something some group of people want. The effort, also, is actually quite small as can be seen on the branch. Perhaps it's even an effort saver if it prevents the topic coming up again and again ;) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
