Re: [Zope-dev] zope.testrunner and nose count doctests differently

2011-11-08 Thread Raphael Ritz
On 11/3/11 12:13 PM, Chris Withers wrote:
 On 03/11/2011 11:05, Jim Fulton wrote:
 On Thu, Nov 3, 2011 at 5:14 AM, Chris Withersch...@simplistix.co.uk   
 wrote:
 Hi Michael,

 On 03/11/2011 09:12, Michael Howitz wrote:
 Run both test runners with the option -vv to see which tests are run.
 (I did this for your code and the list of tests seems to be equal.)

 It would be interesting for the rest of us to know the length of that list.

 $ bin/test -m testfixtures.tests.test_docs -vv
 Running tests at level 1
 Running zope.testrunner.layer.UnitTests tests:
 Set up zope.testrunner.layer.UnitTests in 0.000 seconds.
 Running:
testfixtures/docs/api.txt
testfixtures/docs/changes.txt
testfixtures/docs/comparing.txt
testfixtures/docs/components.txt
testfixtures/docs/datetime.txt
testfixtures/docs/description.txt
testfixtures/docs/development.txt
testfixtures/docs/exceptions.txt
testfixtures/docs/files.txt
testfixtures/docs/index.txt
testfixtures/docs/installation.txt
testfixtures/docs/license.txt
testfixtures/docs/logging.txt
testfixtures/docs/mocking.txt
testfixtures/docs/streams.txt
testfixtures/docs/utilities.txt


Note that the above are 16 files ...

 Ran 316 tests with 0 failures and 0 errors in 0.150 seconds.

... containing what zope.testrunner considers 316 individual tests.

 Tearing down left over layers:
 Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.

 $ bin/nosetests testfixtures/tests/test_docs.py -vv
 nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
 testfixtures_nose/docs/api.txt ... ok
 testfixtures_nose/docs/changes.txt ... ok
 testfixtures_nose/docs/comparing.txt ... ok
 testfixtures_nose/docs/components.txt ... ok
 testfixtures_nose/docs/datetime.txt ... ok
 testfixtures_nose/docs/description.txt ... ok
 testfixtures_nose/docs/development.txt ... ok
 testfixtures_nose/docs/exceptions.txt ... ok
 testfixtures_nose/docs/files.txt ... ok
 testfixtures_nose/docs/index.txt ... ok
 testfixtures_nose/docs/installation.txt ... ok
 testfixtures_nose/docs/license.txt ... ok
 testfixtures_nose/docs/logging.txt ... ok
 testfixtures_nose/docs/mocking.txt ... ok
 testfixtures_nose/docs/streams.txt ... ok
 testfixtures_nose/docs/utilities.txt ... ok

These are the same 16 files (I assume) that nose considers one
test each irrespective of their content. The individual tests
in there are run but not counted separately.

Is it this what's confusing you? It certainly confused me
when I noticed this for the first time. I'd even consider
this a bug in node's reporting but never dared to file a
ticket.

Raphael


 --
 Ran 16 tests in 0.141s

 My guess is that when you're using the zope testruner, you're somehow
 picking up zope.testing.doctest, which counts each doctest example as
 a  test.

 The code uses Manuel, under both nose and zope.testrunner:

 from doctest import REPORT_NDIFF,ELLIPSIS
 from glob import glob
 from manuel import doctest,codeblock,capture
 from manuel.testing import TestSuite
 from os.path import dirname,join,pardir

 def test_suite():
   m =  doctest.Manuel(optionflags=REPORT_NDIFF|ELLIPSIS)
   m += codeblock.Manuel()
   m += capture.Manuel()
   return TestSuite(
   m,
   *glob(join(dirname(__file__),pardir,pardir,'docs','*.txt'))
   )

 cheers,

 Chris



___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] zope-tests - FAILED: 2, OK: 45

2011-11-08 Thread Zope tests summarizer
This is the summary for test reports received on the 
zope-tests list between 2011-11-07 00:00:00 UTC and 2011-11-08 00:00:00 UTC:

See the footnotes for test reports of unsuccessful builds.

An up-to date view of the builders is also available in our 
buildbot documentation: 
http://docs.zope.org/zopetoolkit/process/buildbots.html#the-nightly-builds

Reports received


   Bluebream / Python2.5.5 64bit linux
   Bluebream / Python2.6.7 64bit linux
   Bluebream / Python2.7.2 64bit linux
   ZTK 1.0 / Python2.4.6 Linux 64bit
   ZTK 1.0 / Python2.5.5 Linux 64bit
   ZTK 1.0 / Python2.6.7 Linux 64bit
   ZTK 1.0dev / Python2.4.6 Linux 64bit
   ZTK 1.0dev / Python2.5.5 Linux 64bit
   ZTK 1.0dev / Python2.6.7 Linux 64bit
   ZTK 1.1 / Python2.5.5 Linux 64bit
   ZTK 1.1 / Python2.6.7 Linux 64bit
   ZTK 1.1 / Python2.7.2 Linux 64bit
   ZTK 1.1dev / Python2.5.5 Linux 64bit
   ZTK 1.1dev / Python2.6.7 Linux 64bit
   ZTK 1.1dev / Python2.7.2 Linux 64bit
   Zope 3.4 KGS / Python2.4.6 64bit linux
   Zope 3.4 KGS / Python2.5.5 64bit linux
[1]Zope 3.4 Known Good Set / py2.4-32bit-linux
   Zope 3.4 Known Good Set / py2.4-64bit-linux
   Zope 3.4 Known Good Set / py2.5-32bit-linux
   Zope 3.4 Known Good Set / py2.5-64bit-linux
   Zope-2.10 Python-2.4.6 : Linux
   Zope-2.11 Python-2.4.6 : Linux
   Zope-2.12 Python-2.6.6 : Linux
   Zope-2.12-alltests Python-2.6.6 : Linux
   Zope-2.13 Python-2.6.6 : Linux
   Zope-2.13-alltests Python-2.6.6 : Linux
   Zope-trunk Python-2.6.6 : Linux
   Zope-trunk-alltests Python-2.6.6 : Linux
   winbot / ZODB_dev py_254_win32
   winbot / ZODB_dev py_265_win32
   winbot / ZODB_dev py_265_win64
   winbot / ZODB_dev py_270_win32
   winbot / ZODB_dev py_270_win64
[2]winbot / zope.testing_py_265_32
   winbot / ztk_10 py_254_win32
   winbot / ztk_10 py_265_win32
   winbot / ztk_10 py_265_win64
   winbot / ztk_11 py_254_win32
   winbot / ztk_11 py_265_win32
   winbot / ztk_11 py_265_win64
   winbot / ztk_11 py_270_win32
   winbot / ztk_11 py_270_win64
   winbot / ztk_dev py_265_win32
   winbot / ztk_dev py_265_win64
   winbot / ztk_dev py_270_win32
   winbot / ztk_dev py_270_win64

Non-OK results
--

[1]FAILED  Zope 3.4 Known Good Set / py2.4-32bit-linux
   https://mail.zope.org/pipermail/zope-tests/2011-November/052357.html


[2]FAILED  winbot / zope.testing_py_265_32
   https://mail.zope.org/pipermail/zope-tests/2011-November/052360.html


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )