Re: [Zope-dev] RFC: adding support for testing from bare setuptools
On Thu, Sep 18, 2008 at 12:42:33AM +0200, Philipp von Weitershausen wrote: > Marius Gedminas wrote: > > On Wed, Sep 17, 2008 at 12:52:49PM -0400, Tres Seaver wrote: > >> $ ../bin/python setup.py test # [2] > >> # Runs egg_info, installs regular and testing dependencies, and > >> # runs all unit (non-layer) tests > > > > I don't like the idea that running the tests installs additional > > packages into my environment without me explicitly asking for it. > > You *are* asking for it by running python setup.py test. I think I got confused (again) by the difference by setuptools and distutils. setuptools has a 'setup.py test' that installs dependencies and runs tests. distutils doesn't have a 'setup.py test' at all. For some reason I thought 'setup.py test' was a distutils thing that didn't know anything about dependencies, and that people who use it were used it not changing the environment. My mistake. > Also, python > setup.py test doesn't actually install testing dependencies (or any > dependencies for that matter) into site-packages. It just dumps them > into the CWD. I don't mind that. Marius Gedminas -- This is the Local Descriptor Table, another weird Intel thingy. Linux only uses this for some strange applications like Wine. We don't do anything here, so they'll get an informative and friendly Segmentation Fault. -- lguest source code signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.sqlalchemy locks up transaction
Brian Sutherland wrote at 2008-9-17 12:33 +0200: >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote: >> Brian Sutherland wrote: >> > Hi, >> > >> > I've recently seen a situation where zope.sqlalchemy locked up the >> > transaction machinery. I'm not sure exactly what happened, but have >> > attached a failing test for at least one bug which may have caused it. >> > Hopefully it's self explanatory;) >> > >> > If someone could help me solve this, that would be great! >> >> Could you try this with latest trunk. I checked in a fix the other day >> that may help. > >I just checked in a fix, please feel free to comment on/revert it if >it's not up to standard:) It looks not yet right to clear the state in "tpc_vote" when a two phase commit is used (which is now supported by "SQLAlchemy"). In addition, there may be a problem in case "session.close()" raises an exception. Then, "_finish" would not be called. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: adding support for testing from bare setuptools
Marius Gedminas wrote: > On Wed, Sep 17, 2008 at 12:52:49PM -0400, Tres Seaver wrote: >> Proposal >> >> >> Instead, what I *am* proposing is adding metadata which allows consumers >> of such packages to verify that the package is downloaded / installed >> correctly. In particular, I want for the following scenario to work >> seamlessly:: >> >> $ /path/to/virtualenv --no-site-packges sandbox >> ... >> $ cd sandbox >> $ tar xzf zope.foo.bar-3.5.6.tar.gz >> $ cd zope.foo.bar-3.5.6 # [1] >> >> $ ../bin/python setup.py test # [2] >> # Runs egg_info, installs regular and testing dependencies, and >> # runs all unit (non-layer) tests > > I don't like the idea that running the tests installs additional > packages into my environment without me explicitly asking for it. You *are* asking for it by running python setup.py test. Also, python setup.py test doesn't actually install testing dependencies (or any dependencies for that matter) into site-packages. It just dumps them into the CWD. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: adding support for testing from bare setuptools
On Wed, Sep 17, 2008 at 12:52:49PM -0400, Tres Seaver wrote: > Proposal > > > Instead, what I *am* proposing is adding metadata which allows consumers > of such packages to verify that the package is downloaded / installed > correctly. In particular, I want for the following scenario to work > seamlessly:: > > $ /path/to/virtualenv --no-site-packges sandbox > ... > $ cd sandbox > $ tar xzf zope.foo.bar-3.5.6.tar.gz > $ cd zope.foo.bar-3.5.6 # [1] > > $ ../bin/python setup.py test # [2] > # Runs egg_info, installs regular and testing dependencies, and > # runs all unit (non-layer) tests I don't like the idea that running the tests installs additional packages into my environment without me explicitly asking for it. > $ ../bin/python setup.py test# [3] > # Runs egg_info, installs regular and t This is truncated. > # runs *all* tests ... > [3] In the second method, we use the setuptools entry point mechanism to > register a custom 'test' command[5]. This is a wee bit fragile, as > running 'setup.py test' without first generating the egg_info will use > the "standard" testrunner (but only the first time). If such fragility > is an issue, we can register a separate command, e.g. 'ftest'; in > either case, we have to document that you need to run 'setup.py > egg_info' one time before running the custom command. (The more I learn about setuptools, the less I like them.) > If the ideal > installation outlined in [1] above worked, that would solve the problem, > too, because the egg-info would be generated by easy_install. > > This techniquie drives the zope.testing.testrunner via an entry point, > which is cool, but it cannot expose all of testrunner's features. > In particular, the configuration knobs which the testrunner expects to > get on the command-line clash with distutils' own command-line syntax > (yes, I've tried to make it work). The command I have provided just > runs all the tests, unit and functional, which seems to be reasonable > assurance for the package conumer that the package installed cleanly. *nod* > Implementation > -- > > The intent is to add the attached module as > 'zope.testing.testrunner.eggsupport', At this point I'm wondering whether we shouldn't split out the testrunner into a separate zope.testrunner package. > and then to modify our packages > (opportunistically, or through a 'geddon) as follows: ... > The second attachment is a diff showing the changes to a typical Zope2 > product's setup.py to support this proposal. I'm +0.5 for the idea, although the amount of ritual dancing in the setup.py seems a bit excessive. Marius Gedminas -- 99 little bugs in the code, 99 bugs in the code, fix one bug, compile it again... 101 little bugs in the code signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.sqlalchemy locks up transaction
Brian Sutherland wrote at 2008-9-16 09:56 +0200: >I've recently seen a situation where zope.sqlalchemy locked up the >transaction machinery. I'm not sure exactly what happened, but have >attached a failing test for at least one bug which may have caused it. >Hopefully it's self explanatory;) > >If someone could help me solve this, that would be great! We have seen similar problems and hope to understand/work around them soon. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2: Unicode object ids and PropertyManager properties
Paul Winkler wrote at 2008-9-12 09:04 -0400:
>Is there a concensus on the best way in zope 2 to handle non-ascii
>object IDs? The current restrictions are based on a very old,
>gradually updated regex which still isn't right, see for example
>https://bugs.launchpad.net/zope2/+bug/143616
There is a branch which gets rid of the restrictions
"http://svn.zope.org/Zope/branches/dm-arbitrary-ids/";.
It awaits "wetting" (special wish of Tres Seaver).
> ...
>Even if that's wrong, the ascii assumption seems to be so widespread
>in Zope 2 that I think, short of a full audit and a comprehensive
>plan, gradually using isinstance(foo, basestring) might just be false
>advertising that leads people into trouble.
Our local Zope got rid of the ASCII restriction several years
ago, mainly to support WebDAV with the same naming conventions
as typical file systems (i.e. including special characters (umlauts)
in names).
We have met only one problem:
MS-WebDAV usually does not change the encoding -- but
some WebDAV operations ("rename", "copy") follow the
recommendation of HTML 4.01 to first "utf-8" encode and then
url encode.
Thus, we had to cope with both a "native" encoding
and an utf-8 encoding.
--
Dieter
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RFC: adding support for testing from bare setuptools
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Asbestos - First, note that I am not proposing to replace the current testing machinery used by Zope developers working on a given package (typically, buildout plus recipes which configure zope.testing.testrunner as a standalone script). Nor am I proposing that we modify that machinery to take advantage of setuptools' test-oriented metadata (that would be a separate proposal). Proposal - Instead, what I *am* proposing is adding metadata which allows consumers of such packages to verify that the package is downloaded / installed correctly. In particular, I want for the following scenario to work seamlessly:: $ /path/to/virtualenv --no-site-packges sandbox ... $ cd sandbox $ tar xzf zope.foo.bar-3.5.6.tar.gz $ cd zope.foo.bar-3.5.6 # [1] $ ../bin/python setup.py test # [2] # Runs egg_info, installs regular and testing dependencies, and # runs all unit (non-layer) tests $ ../bin/python setup.py test# [3] # Runs egg_info, installs regular and t # runs *all* tests Theory of Operation - --- [1] Ideally, this would be some variation on 'easy_install' which got the package into the environment in such a way that its 'setup.py' could be run. Fetching and unpacking the tarball manually works today, however ;) [2] 'python setup.py test' is the "standard" way to test a setuptools-based package, and would be expected to work by most egg-savvy Python developers. setuptools provides the 'test' command, using a built-in testrunner which mostly Just Works(TM). However, it doesn't support the 'layer' feature provided by 'zope.testrunner', and therefore many Zopeish packages emit failures when run this way. Without monkeypatching setuptools, we can't replace the testrunner it uses, but we *can* supply a different test loader using an option in setup.py[4]. The attached module provides such a loader, which filters out the doomed tests, leaving only those which can run cleanly without layers (e.g., the unit tests). [3] In the second method, we use the setuptools entry point mechanism to register a custom 'test' command[5]. This is a wee bit fragile, as running 'setup.py test' without first generating the egg_info will use the "standard" testrunner (but only the first time). If such fragility is an issue, we can register a separate command, e.g. 'ftest'; in either case, we have to document that you need to run 'setup.py egg_info' one time before running the custom command. If the ideal installation outlined in [1] above worked, that would solve the problem, too, because the egg-info would be generated by easy_install. This techniquie drives the zope.testing.testrunner via an entry point, which is cool, but it cannot expose all of testrunner's features. In particular, the configuration knobs which the testrunner expects to get on the command-line clash with distutils' own command-line syntax (yes, I've tried to make it work). The command I have provided just runs all the tests, unit and functional, which seems to be reasonable assurance for the package conumer that the package installed cleanly. Implementation - -- The intent is to add the attached module as 'zope.testing.testrunner.eggsupport', and then to modify our packages (opportunistically, or through a 'geddon) as follows: - Add 'setup_requires = ["eggtestinfo"]', so that the testing metadata is captured in EGG-INFO. Note that this information is not needed when running the tests from a source distribution, but should ease a TBD mode for testing binary distributions. - Add 'tests_require = ["zope.testing >= 3.7"]' (or whatever the next released version including this feature would be), to make the loader available. - Add 'test_loader = "zope.testing.testrunner.eggsupport:SkipLayers"' which enables 'setup.py test' to use the custom loader. Once the new 'zope.testing' package is present, then the 'ftest' command will also be available, but still needs to be called out via an entry point:: entry_points=""" [distutils.commands] ftest = zope.testing.testrunner.eggsupport:ftest """ The second attachment is a diff showing the changes to a typical Zope2 product's setup.py to support this proposal. References - -- [4] http://peak.telecommunity.com/DevCenter/setuptools#test-loader [5] http://peak.telecommunity.com/DevCenter/setuptools#adding-commands Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI0TXh+gerLs4ltQ4RAuEGAJ411zdUI+GjQZEkPtf2uB1zgX3KeQCgvjlA rHFCcXsSaPpY9AklWGqlmqk= =PbVE -END PGP SIGNATURE- eggsupport.py Description: application/httpd-cgi Index: setup.py ==
[Zope-dev] Zope Tests: 2 OK, 3 Failed
Summary of messages to the zope-tests list. Period Tue Sep 16 11:00:00 2008 UTC to Wed Sep 17 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Test failures - Subject: FAILED (failures=1) : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:54:07 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010177.html Subject: FAILED (failures=1) : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:55:37 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010178.html Subject: FAILED (failures=1) : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:57:07 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010179.html Tests passed OK --- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Tue Sep 16 20:51:07 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010175.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:52:37 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010176.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.sqlalchemy locks up transaction
Brian Sutherland wrote: > On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote: >> Brian Sutherland wrote: >>> Hi, >>> >>> I've recently seen a situation where zope.sqlalchemy locked up the >>> transaction machinery. I'm not sure exactly what happened, but have >>> attached a failing test for at least one bug which may have caused it. >>> Hopefully it's self explanatory;) >>> >>> If someone could help me solve this, that would be great! >> Could you try this with latest trunk. I checked in a fix the other day >> that may help. > > I just checked in a fix, please feel free to comment on/revert it if > it's not up to standard:) Great, I'll take a look at it and do a release some time this week. Laurence ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.sqlalchemy locks up transaction
On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote: > Brian Sutherland wrote: > > Hi, > > > > I've recently seen a situation where zope.sqlalchemy locked up the > > transaction machinery. I'm not sure exactly what happened, but have > > attached a failing test for at least one bug which may have caused it. > > Hopefully it's self explanatory;) > > > > If someone could help me solve this, that would be great! > > Could you try this with latest trunk. I checked in a fix the other day > that may help. I just checked in a fix, please feel free to comment on/revert it if it's not up to standard:) > > Laurence > > ___ > Zope-Dev maillist - Zope-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
