Re: [Zope3-dev] Automated compatibility tests

2006-02-09 Thread Stephan Richter
On Wednesday 08 February 2006 07:53, Max M wrote:
 Does anybody have an idea as to what is necessary to do these automated
 tests, or links to previous efforts in this direction? I imagine
 something like the testrunner that is currently testing out Zope.

 Or am I the only one seing this as a problem that should be automated?

I am in the process of writing a proposal that should address this issue for a 
set of fundamental packages.

I really hope that eggs will solve this issue for us eventually.

Regards,
Stephan
-- 
Stephan Richter
CBU Physics  Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Automated compatibility tests

2006-02-08 Thread Tarek Ziadé

Max M wrote:

Being a long time Product developer for Zope 2, I constantly have a 
big problem with checking compatibility for my products with different 
Python-Zope-Plone-Product combinations.


It is also impossible to guess from the Products on zope.org and 
Plone.org which versions works with what.



As far as I can see, the only reasonable solution to this is to 
automate the task with some kind of testrunner.


I can make this testrunner for my own products. But it would be far 
better if there was a standard way to do it, so that it is possible to 
check different version with each other across developers and platforms.



It could be nice to avoid this same problem in Zope 3, or perhaps even 
to find a way to retrofit it to Zope 2.



If I develop a product to run on Zope 3.2, and my customers are using 
this product, I might not test it on 3.3 or 3.4 later on, as I don't 
really have any reason to. So the product could be broken without 
anybody knowing it before trying it out.


If there where automated compatibility tests it would be possible to 
see when a product was no longer usable.


If I then were to update my product, I would also be able to se if my 
new changes would break compatibility with older versions.


It would also be a way for core developers to test if their changes 
in their next Zope version breaks anything in 3. party products. That 
would at least give them a chance to decide to work around it.


The test should include unittest, functional test, migrations of 
content from older product versions etc.



But I imagine that we would need some kind of meta framework for 
setting up tests on a product version per zope version basis.


A result like this would be sweet for everybody:

Zope version3.23.33.4
myProduct 1.2   x  x
myProduct 1.1x  x
myProduct 1.0x


Does anybody have an idea as to what is necessary to do these 
automated tests, or links to previous efforts in this direction? I 
imagine something like the testrunner that is currently testing out Zope.


Or am I the only one seing this as a problem that should be automated?



I think buildbot fills this need very well,
like http://buildbot.zope.org/ or http://buildbot.nuxeo.org/

Tarek
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Automated compatibility tests

2006-02-08 Thread Chris Withers

Tarek Ziadé wrote:

Or am I the only one seing this as a problem that should be automated?


I think buildbot fills this need very well,
like http://buildbot.zope.org/ or http://buildbot.nuxeo.org/


Nah, Max is after something different, and I agree with him that it 
would be great if it could be done automatically, but I suspect it'll 
best be done by hand...


Just 'cos the tests pass doesn't mean it works ;-)

cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Automated compatibility tests

2006-02-08 Thread Tarek Ziadé

Chris Withers wrote:


Tarek Ziadé wrote:


Or am I the only one seing this as a problem that should be automated?


I think buildbot fills this need very well,
like http://buildbot.zope.org/ or http://buildbot.nuxeo.org/



Nah, Max is after something different, and I agree with him that it 
would be great if it could be done automatically, but I suspect it'll 
best be done by hand...


Just 'cos the tests pass doesn't mean it works ;-)


check out your coverage score then ;)



cheers,

Chris




___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com