Re: [Zope-dev] testing everything
On 2009-08-18, Hanno Schlichting ha...@hannosch.eu wrote:
On Tue, Aug 18, 2009 at 4:07 PM, Reinout van Reesrein...@vanrees.org wrote:
Question: is there an existing recipe that does this? Main point is that I
don't want to specify the dependencies that are to be tested by hand, so that
seems to rule out z3c.recipe.compattest. But there's been so much talk about
testing everything in the last weeks that I thought it better to ask :-)
I wrote http://pypi.python.org/pypi/plone.recipe.alltests, which tries
to do something similar at least.
Ha! That works fine and we're going to use it.
I *did* get a bit scared of all the errors you get in zope.app.* packages.
But that's apparently a known problem (and something that can be suppressed by
the exclude parameter).
What do you think about adding defaults for the eggs and test-script
parameters? Defaulting them ${test:eggs} and bin/test is probably enough in
99% of the cases. If you're OK with that I'll try it in a branch.
Reinout
--
Reinout van Rees - rein...@vanrees.org - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
Military engineers build missiles. Civil engineers build targets
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] testing everything
On Wed, Aug 19, 2009 at 10:40 AM, Reinout van Reesrein...@vanrees.org wrote:
On 2009-08-18, Hanno Schlichting ha...@hannosch.eu wrote:
I wrote http://pypi.python.org/pypi/plone.recipe.alltests, which tries
to do something similar at least.
Ha! That works fine and we're going to use it.
Awesome :)
I *did* get a bit scared of all the errors you get in zope.app.* packages.
But that's apparently a known problem (and something that can be suppressed by
the exclude parameter).
The main intent for the exclude argument is to exclude packages which
cannot be tested either from their shipped distribution or are not
zope.testing compatible like pytz, Paste, WebOb or similar.
If there's failures in zope.app these should be fixed ;)
What do you think about adding defaults for the eggs and test-script
parameters? Defaulting them ${test:eggs} and bin/test is probably enough in
99% of the cases. If you're OK with that I'll try it in a branch.
Reinout made a branch and these changes are soon available in a new release.
Hanno
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] testing everything
On 2009-08-19, Hanno Schlichting ha...@hannosch.eu wrote:
On Wed, Aug 19, 2009 at 10:40 AM, Reinout van Reesrein...@vanrees.org wrote:
I *did* get a bit scared of all the errors you get in zope.app.* packages.
But that's apparently a known problem (and something that can be suppressed
by the exclude parameter).
The main intent for the exclude argument is to exclude packages which
cannot be tested either from their shipped distribution or are not
zope.testing compatible like pytz, Paste, WebOb or similar.
If there's failures in zope.app these should be fixed ;)
Yep. There was a some discussion here about that (in a thread about running
all tests on the ZTK iirc) that the combination of tests was in a sad state.
jdriessen suggested that might be something to sort out partially at the
upcoming grok sprint.
What do you think about adding defaults for the eggs and test-script
parameters? Defaulting them ${test:eggs} and bin/test is probably enough in
99% of the cases. If you're OK with that I'll try it in a branch.
Reinout made a branch and these changes are soon available in a new release.
Yep, they're released. Thanks!
Reinout
--
Reinout van Rees - rein...@vanrees.org - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
Military engineers build missiles. Civil engineers build targets
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] testing everything
In many buildouts I have a [test-all] section. Just like [test], but then with more eggs (typically all the dependencies). That's too much work, so I'm looking for a recipe to handle that for me. My thoughts: - Subclass zc.recipe.testrunner in a new recipe. - Look for a [test] part in buildout and take the defaults there. - Examine the eggs in test:eggs and look up their setup.py dependencies. - Those dependencies, that's the new list of eggs that need testing. - Just a [test-all] with the new recipe should be enough to test all main dependencies. Question: is there an existing recipe that does this? Main point is that I don't want to specify the dependencies that are to be tested by hand, so that seems to rule out z3c.recipe.compattest. But there's been so much talk about testing everything in the last weeks that I thought it better to ask :-) Reinout -- Reinout van Rees - rein...@vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com Military engineers build missiles. Civil engineers build targets ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] testing everything
On Tue, Aug 18, 2009 at 4:07 PM, Reinout van Reesrein...@vanrees.org wrote:
Question: is there an existing recipe that does this? Main point is that I
don't want to specify the dependencies that are to be tested by hand, so that
seems to rule out z3c.recipe.compattest. But there's been so much talk about
testing everything in the last weeks that I thought it better to ask :-)
I wrote http://pypi.python.org/pypi/plone.recipe.alltests, which tries
to do something similar at least.
It's main assumption is to test everything found in the current
working set and let you specify some excludes. A somewhat minimal
example would look something like this:
[alltests]
recipe = plone.recipe.alltests
eggs = ${test:eggs}
exclude =
zope.foobar
test-script = ${buildout:bin-directory}/test
Where test would be a regular zc.recipe.testrunner section.
The integration is currently crude, in that it just calls the existing
test script via os.system instead of doing any direct integration.
That also causes the redundancy of specifying both the eggs and the
test-script. This could be replaced by a pointer to the test
section.
The recipe also defaults to running the tests for each package on its
own and has an explicit notion of grouping multiple packages into one
test group that is known to run together without side effects. In
Plone testing land the default assumption is, that there is generally
side-effects while testing. For ZTK packages it might be ok to assume
that generally all tests of all packages work without side effects and
only some packages might need to be isolated.
I'd be happy to see patches to the recipe, it's in the collective and
ZPL licensed.
Hanno
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
