Re: [Zope-dev] Test runner: layers, subprocesses, and tear down
Hi, On Thu, 2008-07-03 at 17:22 -0400, Benji York wrote: I'm working on making the zope.testing test runner run tests in parallelized subprocesses. The option will likely be spelled -j N, where N is the maximum number of processes. Getting back to the idea about parallelizing on a per-test base and not per-layer: The ZODB currently runs only unit tests (which became a true layer in zope.testing/trunk) but takes about XX minutes on one of my machines (4 core XEON, 3.2 GHz). I'd suggest that the general principle of splitting up the runs over multiple parallel processes should happen in a way that if you have X total tests and N parallel processes, we should have roughly X/N tests run in parallel. We could use layers as a hint to create subprocesses, but should split up layers if they are too large to fit the X/N rule (maybe with a margin of a few percent to avoid splits for single or few tests). Christian -- Christian Theune · [EMAIL PROTECTED] gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development signature.asc Description: This is a digitally signed message part ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test runner: layers, subprocesses, and tear down
On Sat, 2008-07-05 at 09:18 +0200, Christian Theune wrote: Hi, On Thu, 2008-07-03 at 17:22 -0400, Benji York wrote: I'm working on making the zope.testing test runner run tests in parallelized subprocesses. The option will likely be spelled -j N, where N is the maximum number of processes. Getting back to the idea about parallelizing on a per-test base and not per-layer: The ZODB currently runs only unit tests (which became a true layer in zope.testing/trunk) but takes about XX minutes on one of my machines (4 core XEON, 3.2 GHz). The actual numbers are: Ran 2816 tests with 0 failures and 0 errors in 14 minutes 34.292 seconds. real14m36.099s user3m44.740s sys 0m43.170s -- Christian Theune · [EMAIL PROTECTED] gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development signature.asc Description: This is a digitally signed message part ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 5 OK
Summary of messages to the zope-tests list. Period Fri Jul 4 11:00:00 2008 UTC to Sat Jul 5 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Fri Jul 4 20:55:41 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009805.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Fri Jul 4 20:57:11 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009806.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Fri Jul 4 20:58:41 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009807.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Fri Jul 4 21:00:12 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009808.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Fri Jul 4 21:01:42 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009809.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test runner: layers, subprocesses, and tear down
On Sat, Jul 5, 2008 at 3:18 AM, Christian Theune [EMAIL PROTECTED] wrote: We could use layers as a hint to create subprocesses, but should split up layers if they are too large to fit the X/N rule (maybe with a margin of a few percent to avoid splits for single or few tests). It probably wouldn't be too hard to automatically break large layers into several small layers. It's also not hard for people with very large layers that care about parallel execution time to break them up themselves. I'm not opposed to automatic layer segmentation (as long as it's implemented well), but also don't think it's all that important. -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 3 on Python 2.5, Zope 3 releases
On Thursday 26 June 2008, Martijn Faassen wrote: I'm curious about the plans of Zope 3 on Python 2.5. * Are people using Zope 3 with Python 2.5 already? What are your experiences? Yes, works flawlessly. The problems in zope.security and zope.proxy that I recently found (and fixed) exist for Python 2.4 as well. * Are there plans for a release after Zope 3.4? I'm just assuming that Zope 3.4.0c1 is actually the only release that Zope 3.4 is going to see. Yeah, yeah. Next week I am traveling, so I plan to just update the packages to the latest bug-fix release and publish it. Any plans or interest in a Zope 3.5? For me the interest is more of having a stable KGS than the TAR ball. * Is someone exercising the trunk version of KGS, running the tests to see whether everything works together? No I have not started doing that yet. Is someone actually updating the versions? No, but producing it is not that much work. Could we somehow broaden the community for this? Yes, please. It effectively means giving more people to the directories on download.zope.org. What would the policy be for checkins of Zope-related packages? Could we use new Python 2.5 features, for instance? That's a good question. I am +1 for that. Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. Zope Stephan Richter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Parallel subprocesses ready to merge
My branch that can (but by default doesn't) run subprocesses in parallel is ready to merge. It's not a panacea (if you have carefully crafted layers that can all be torn down, your test runs will probably be slower with the feature switched on), but for the common -- I believe -- case of having several layers that can't be torn down, tests run much faster on multi-core machines. The branch is svn+ssh://svn.zope.org/repos/main/zope.testing/branches/benji-parallelize-subprocesses. If no one finds major problems with the branch by the middle of next week, I'll merge it to the trunk. -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zc.testbrowser.real support for mozlab 0.1.9
On 4 Jul 2008, at 17:34, Christian Zagrodnick wrote:
On 2008-07-04 01:19:47 +0200, Graham Stratton [EMAIL PROTECTED]
beasts.com said:
I tried running the tests with Python 2.4. About half the time
they pass fine, and the other half fail something like this:
File doctest README.txt[40], line 1, in ?
browser.open('navigate.html')
File /Users/graham/development/patched_zc.testbrowser/src/
zc/ testbrowser/real.py, line 201, in open
self.wait()
File /Users/graham/development/patched_zc.testbrowser/src/
zc/ testbrowser/real.py, line 193, in wait
assert self.execute('tb_page_loaded;') == 'false'
AssertionError
though for varying pages with browser.open(). I don't understand
what could be causing it; why would that value be changed in
between those two lines? I guess the time.sleep() on line 191
implies that there's something we might want to wait for. I
increased it by 10x and only got 1 failure out of 10, so I guess
that helps.
I had this error, too and checked in a maybe-fix in r87964. Have you
tried with this revision or the one before? If it's thisone I'll add
another loop there.
I've tried with this revision, and with the timeout extended, but
still occasionally see the problem.
I have a couple of independent fixes for other issues which I'll
discuss once this branch is merged.
Regards,
Graham
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
