Hash: SHA1

- --------

First, note that I am not proposing to replace the current testing
machinery used by Zope developers working on a given package (typically,
buildout plus recipes which configure zope.testing.testrunner as a
standalone script).  Nor am I proposing that we modify that machinery to
take advantage of setuptools' test-oriented metadata (that would be a
separate proposal).

- --------

Instead, what I *am* proposing is adding metadata which allows consumers
of such packages to verify that the package is downloaded / installed
correctly.  In particular, I want for the following scenario to work

 $ /path/to/virtualenv --no-site-packges sandbox
 $ cd sandbox
 $ tar xzf zope.foo.bar-3.5.6.tar.gz
 $ cd zope.foo.bar-3.5.6           # [1]

 $ ../bin/python setup.py test     # [2]
 # Runs egg_info, installs regular and testing dependencies, and
 # runs all unit (non-layer) tests

 $ ../bin/python setup.py test    # [3]
 # Runs egg_info, installs regular and t
 # runs *all* tests

Theory of Operation
- -------------------

[1] Ideally, this would be some variation on 'easy_install' which got
the package into the environment in such a way that its 'setup.py' could
be run.  Fetching and unpacking the tarball manually works today, however ;)

[2] 'python setup.py test' is the "standard" way to test a
setuptools-based package, and would be expected to work by most
egg-savvy Python developers.  setuptools provides the 'test' command,
using a built-in testrunner which mostly Just Works(TM).  However, it
doesn't support the 'layer' feature provided by 'zope.testrunner', and
therefore many Zopeish packages emit failures when run this way.

Without monkeypatching setuptools, we can't replace the testrunner it
uses, but we *can* supply a different test loader using an option in
setup.py[4].  The attached module provides such a loader, which filters
out the doomed tests, leaving only those which can run cleanly without
layers (e.g., the unit tests).

[3] In the second method, we use the setuptools entry point mechanism to
register a custom 'test' command[5].  This is a wee bit fragile, as
running 'setup.py test' without first generating the egg_info will use
the "standard" testrunner (but only the first time).  If such fragility
is an issue, we can register a separate command, e.g. 'ftest';  in
either case, we have to document that you need to run 'setup.py
egg_info' one time before running the custom command.  If the ideal
installation outlined in [1] above worked, that would solve the problem,
too, because the egg-info would be generated by easy_install.

This techniquie drives the zope.testing.testrunner via an entry point,
which is cool, but it cannot expose all of testrunner's features.
In particular, the configuration knobs which the testrunner expects to
get on the command-line clash with distutils' own command-line syntax
(yes, I've tried to make it work).  The command I have provided just
runs all the tests, unit and functional, which seems to be reasonable
assurance for the package conumer that the package installed cleanly.

- --------------

The intent is to add the attached module as
'zope.testing.testrunner.eggsupport', and then to modify our packages
(opportunistically, or through a 'geddon) as follows:

 - Add 'setup_requires = ["eggtestinfo"]', so that the testing metadata
   is captured in EGG-INFO.  Note that this information is not needed
   when running the tests from a source distribution, but should ease
   a TBD mode for testing binary distributions.

 - Add 'tests_require = ["zope.testing >= 3.7"]' (or whatever the next
   released version including this feature would be), to make the
   loader available.

 - Add 'test_loader = "zope.testing.testrunner.eggsupport:SkipLayers"'
   which enables 'setup.py test' to use the custom loader.

Once the new 'zope.testing' package is present, then the 'ftest' command
will also be available, but still needs to be called out via an entry

   ftest = zope.testing.testrunner.eggsupport:ftest

The second attachment is a diff showing the changes to a typical Zope2
product's setup.py to support this proposal.

- ----------

[4] http://peak.telecommunity.com/DevCenter/setuptools#test-loader

[5] http://peak.telecommunity.com/DevCenter/setuptools#adding-commands

- --
Tres Seaver          +1 540-429-0999          [EMAIL PROTECTED]
Palladion Software   "Excellence by Design"    http://palladion.com
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


Attachment: eggsupport.py
Description: application/httpd-cgi

Index: setup.py
--- setup.py	(revision 91203)
+++ setup.py	(working copy)
@@ -43,17 +43,21 @@
-      tests_require=[
-          "five.localsitemanager >= 0.2",
+      setup_requires=['eggtestinfo',
+      tests_require=['zope.testing',
+                     'five.localsitemanager >= 0.2',
+                    ],
-      install_requires=[
-          "setuptools",
-          "five.localsitemanager >= 0.2",
-#          'Zope >= 2.10',
+      test_loader="zope.testing.testrunner.eggsupport:SkipLayers",
+      install_requires=['setuptools',
+                        'five.localsitemanager >= 0.2',
+                       #'Zope2-buildout >= 2.10',
       Products.GenericSetup = Products.GenericSetup:initialize
+      [distutils.commands]
+      test = zope.testing.testrunner.eggsupport:ftest
Zope-Dev maillist  -  Zope-Dev@zope.org
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope )

Reply via email to