Re: [Zope-dev] Uses of the ZTK and how it relates to management
On 03/04/2010 11:35 AM, Hanno Schlichting wrote: On Thu, Mar 4, 2010 at 7:19 AM, Christian Theune c...@gocept.com wrote: A thought that came up when reading this paragraph: another option restructuring/grouping to reduce the amount of packages may be to join smaller packages with weird boundaries into larger ones again. (Not that I suggest this to be an ultimate option, nor do I know from the top of my head any candidates for this, but we can keep that on the list of options we have.) I think this is a good idea, but I wouldn't want to do it on a package level. Rather do it on the distribution level. Once the distutils2 improvements are available, we have the means to say distribution A obsoletes B. Interesting. On the first impression that sounds like a good abstraction and tool. I guess not everyone aware that distribution != package is possible and maybe even useful. ;) Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development smime.p7s Description: S/MIME Cryptographic Signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Summary of today's developer meeting
Hi, On 03/04/2010 09:17 PM, Sebastien Douche wrote: On Tue, Mar 2, 2010 at 17:32, Christian Theune c...@gocept.com wrote: here's my first shot at a summary of today's meeting. I found the meeting itself very positive and energetic - thanks again to everyone who joined. The current state of nightly builds is a bit untidy. According to http://docs.zope.org/zopetoolkit/process/buildbots.html there's four buildbot installations with various scopes. The last two in this listing are currently non-functional. The new URL is : http://bluebream.buildbot.securactive.org/ Other buildbots : http://grok.buildbot.securactive.org/ http://bfg.buildbot.securactive.org/ http://misc.buildbot.securactive.org/ Thanks. I will list those in the ZTK documentation. Get a volunteer who will oversee our buildbot installations. The job description would mainly include coordination efforts: ensuring consistent configuration, visibility, reporting and helping people to get nightly builds or contribute builders. mgedmin is pondering until next week whether he volunteers. Funny, it's the same thing each year (read the Message-ID: 20090616060020.gc9...@elzar.ws.whq.gocept.com for a example), developers are not affected by the buildbot state and when I (or other) send a mail about failures, the response is : don't spam the mailing list. Actually, there's a specific aggregation thing (who's running this again?) that can accept the individual buildbot messages and creates a daily message. I think that actually works quite well. I think all the various buildbots should send their messages to this aggregator. BTW I'm a volunteer. Great! We need to put down a list of projects (Zope 2, grok, BB, ZTK, ...), branches and platforms (64-bit!) which we want the nightly builds to be executed on/for. Alan Runyan offered supporting Windows builds. No action/responsibility was agreed upon for this. I have Linux (32 64 bits) Buildbots for all Zope3 projects (grok, bfg, ztk, bb) Cool. I'm also expanding the documentation of what is tested where and who's the contact for the individual buildbots. Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development smime.p7s Description: S/MIME Cryptographic Signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Summary of today's developer meeting
Hi, On 03/05/2010 02:56 AM, Baiju M wrote: Hi Sebastien, On Fri, Mar 5, 2010 at 1:47 AM, Sebastien Douche sdou...@gmail.com wrote: The current state of nightly builds is a bit untidy. According to http://docs.zope.org/zopetoolkit/process/buildbots.html there's four buildbot installations with various scopes. The last two in this listing are currently non-functional. The new URL is : http://bluebream.buildbot.securactive.org/ Thanks for setting up Buildbot for BB. Can you please co-ordinate with Christophe Combelles, he already setup a Buildbot for BB: http://zope3.afpy.org/buildbot/waterfall (I have CC'ed him here) Baiju : Do you want another Buildbots? (za = zopeproject, com = community, ztk = Zope TollKit, bb = BlueBream). A single build master as you organized looks good. Do we need multiple masters located at different servers ? What I really needed is more slaves. We need to find volunteers to provide Windows slaves (both 32bit 64bit) I'm currently making the ZTK documentation a point where all the efforts at least get listed so we see who takes care of what. I'm not pressing a single master so we get better coverage because more people can directly influence their setups. For now that probably makes the best out of that energy. :) With the overview you should be able to talk to the maintainers of those buildbots and ask them whether they can help you. Enfold seems to be happy to provide Windows environments generally. I also noticed that the health agency runs Windows tests. Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Automating tests of the ZTK / zopeapp package sets
Hi, I also looked at Hudson a few days ago and found it to be awesome. :) On 03/04/2010 10:10 PM, Tres Seaver wrote: After successfully configuring the Hudson[1] continuous integration server yesterday to test the various repoze.* packages[2], I thought I would experiment with using it to drive the compatibility tests for ZTK and zopeapp. Here is what I did: In a shell on my server: $ wget http://hudson-ci.org/latest/hudson.war $ java -jar hudson.war --httpPort=1 That's what blew me off my chair. :) In the browser, visit 'http://laguna.palladion.com:1, and configure the server: - Enable security - allow signup - using Matrix - set up 'tseaver' as having all permissions - set up 'anonymous' as having 'Read' permission overall and for jobs - Register as 'tseaver' - Turn off signup (no, I don't want you guys running arbitrary shell scripts on my server ;). - Create a job, 'ztk-trunk-py26': See below http://laguna.palladion.com:1/job/ztk-trunk-py26/ - Use SVN, pointed at 'svn://svn.zope.org/repos/main/zopetoolkit/trunk' - Set to poll SVN every 15 minutes. - Enable using 'svn up' - Added build step as a shell script:: cd trunk /opt/Python-2.6.4/bin/python bootstrap.py bin/buildout bin/test-ztk bin/test-zopeapp - Ran a build manually. The first one barfed due to a typo in my script, but the second one ran, taking 21 minutes. Right. That was my experience as well. Also a good thing is that you can now take those as templates, or - if you like - generate those XML files into the filesystem for new builds and ping the server to reload those. The test runs successfully to completion, but doesn't produce statistics in a form that Hudson understands. The following changes to 'zope.testing.testrunner' would make the output more informative: - Add a flag like the '--with-xunit' flag to the Nose testrunner, which dumped results into a JUnit-compatible XML file[3]. - Add a flag like the '--with-xcompat' flag to the Nose testrunner (when run with the 'nosexcover' plugin), producing a Cobertura- compabtible XML file[4]. Right. However Hudson does understand failures/successes at least. I made a job for the ZODB where the individual steps (checkout, bootstrap/buildout, test) where separated steps so that made the UI integration a little bit better. At that point, the build process would be nicely automated, with browsable results including coverage, pretty graphs, etc. See the 'repoze' jobs for examples of those outputs. Here's another awesome part: You can configure matrix build jobs which would allow us to define a single job for i.e. ZODB and then say we want this tun run in the combinations of Unix/Windows, 32/64, Python 2.4/2.5/2.6, trunk/branchx /branchy and it goes off automatically trying to find build nodes that can provide all combinations. It took my about 10 minutes to get that going including the installment of the Windows builder. Questions for discussion: - I find the prettier interface and easier setup than buildbot worth running a 3rd-party Java app (rather than a 3rd party Python app). Would that be acceptable among the folks running our automated tests? It would be for me. Right now I'll continue to clean up what we have with buildbot. I really don't want to put anybody off. Getting more coverage (however) and making it more visible and regular is my primary goal. I don't think we have to through anything out. I'll be happy to provide even more tests using other build systems in parallel. - Is it worth adding the XML support for test results and coverage to 'zope.testing.testrunner'? Maybe. I'm happy hacking on the testrunner. :) - Would people be willing to run Hudson permanently on enough hosts to make this a reasonable replacement for buildbot (we need Windows, too)? I'm definitely willing to run it for 32/64 bit Linux machines. I'm short on Windows licenses but Hudson seems easy to integrate securely distributed over multiples sites and easy enough to set up with the JNRE/service installer. - Hudson seems general purpose enough to allow automating other code- related tasks, e.g., production of dependency graphs. Maybe we could even automate building Windows installers? How awesome that would be. :) I will try to leave the ZTK Hudson server up for a little bit, to allow folks to see what it can deliver already. [1] http://hudson-ci.org/ [2] http://hudson.repoze.org/ [3] http://old.nabble.com/schema-for-junit-xml-output-td22193385.html [4] http://cobertura.sourceforge.net/xml/coverage-01.dtd Thanks a lot for the effort. I'm really excited about Hudson and I'm willing to help out here. Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889
Re: [Zope-dev] Zope3 sessions and database conflicts
On 03/04/2010 11:08 PM, Ross Patterson wrote: Hermann Himmelbauer du...@qwer.tk writes: Hi, For quite some time I see messages like this in my z3.log: 2010-03-02T16:27:14 WARNING ZopePublication Competing writes/reads at /BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/images/sponsor_logo.png: database conflict error (oid 0x063f, class BTrees.OOBTree.OOBucket, serial this txn started with 0x038484dc7d5ac044 2010-03-02 14:52:29.379960, serial currently committed 0x038484ff3c6b5455 2010-03-02 15:27:14.160763) I note the conflict happened when the request was just for an image. I've seen this before under Zope2 when a PAS plugin was mis-using the session machinery as a cache during authentication of the request. When ever a session object is accessed it initiates a write transaction in the ZODB. With most, if not all, authentication methods, when a user is logged-in the auth tokens are included in every request which means every request from the logged-in user invokes the authentication machinery, including requests for images. Since every page load involves multiple requests for page resources, the database gets overwhelmed with write transactions which inevitably conflict. When the write conflict occurs, the publisher appropriately retries the request which multiplies the number of requests which multiplies the load which increases the amount of time taken in processing requests which multiplies the likelihood of write conflicts and your off to the races in the wrong direction. So I'd suggest you find out what in a request for a simple resource (images, non-dynamic CSS or JS) is initiating the write transaction by invoking sessions. It's likely there's an inappropriate use of sessions there. I remember one thing about sessions where just the access to the session would create a new session even if it was never touched. The API actually provides two invokations: one to get a session and create if not existing, the other got get a session and raise a KeyError instead of creating a new one. -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Possible DateTime timezone-related regression in Zope 2.12
On 11.01.10 01:47, Martin Aspeli wrote: Laurence Rowe wrote: I believe the current behaviour is intentional to preserve backwards compatibility. See the discussion starting here: https://mail.zope.org/pipermail/zope-dev/2007-October/030042.html Maybe it was 'fixed' on 2.10 branch some time later. Sorry, just to be clear - which behaviour is correct? The 2.10 one or the 2.12 one? imho, the one from 2.10. i think about anyone would expect to have a simple date given being interpreted as from their current time zone, no? also, look at the following (using `DateTime` 2.12): $ ~/plone/coredev/branches/4.0/bin/zopepy from DateTime import DateTime now = DateTime() now == DateTime(now) True now == DateTime(now.ISO()) False this is a _pretty_ behaviour, to say the least! :) My vote would go for the 2.10 one - in the absence of timezone information, assume local timezone, not GMT. +1, definitely. If we agree on that, is it clear what needs to be changed for this to work? yes, i believe it is. we need to revert Laurence' revert (http://zope3.pov.lt/trac/changeset/81213) and fix those BBB issues in another way. Can we also agree that it's very bad for 2.10 and 2.12 to exhibit different behaviour here? +1 andi -- zeidler it consulting - http://zitc.de/ - i...@zitc.de friedelstraße 31 - 12047 berlin - telefon +49 30 25563779 pgp key at http://zitc.de/pgp - http://wwwkeys.de.pgp.net/ plone 4.0 alpha released! -- http://plone.org/products/plone/ ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Uses of the ZTK and how it relates to management
On 03/04/2010 04:13 PM, Jim Fulton wrote: On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella kob...@kobold.it wrote: * 2010-03-03 21:44, Jim Fulton wrote: The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK. It would be great if that were true. That's been my understanding all the time. I think we currently don't do it because it might be non-obvious to the individual developers at the time of check-in and due to underused tools. We may not require that a single developer run all test suites in all combinations we desire to be compatible, but with the ongoing effort to improve the nightly/automated builds we should get into better shape there. The one issue with delayed testing is that we would have to impose a rule which requires cool down time after a checkin before making a release. That's probably not a terrible thing anyway. Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Uses of the ZTK and how it relates to management
Hi, On 03/04/2010 10:44 PM, Jim Fulton wrote: On Thu, Mar 4, 2010 at 2:37 PM, Tres Seaver tsea...@palladion.com wrote: Weird. Tres' mail didn't make it into gmane ... I'll fold my replies to Jim and Tres into this one mail. (Looks like I'm now turning into the one making noise here. I'm already sorry for people having to read so much. :) ) -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jim Fulton wrote: On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella kob...@kobold.it wrote: * 2010-03-03 21:44, Jim Fulton wrote: The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK. It would be great if that were true. If so, then the recent arguments have been a terrible misunderstanding. I think that most of the misunderstanding lies in the nature of ownership of the packages in the ZTK: some would like everyone to care equally about all the packages (including those now factored into the zopeapp.cfg set), while others want to give most of their attention to some smaller set of packages which they use every day. Something that appeared to me while reading that: the specific set that a single developer cares for varies over time. Independent of the specific set I care for at any point in time, I still am interested in a healthy larger set because I might need that again tomorrow or next week. We already have some policies in place which recognize that the bicycle seat toolkit packages need special handling, becaues they are used outside the Zope ecosystem (one rule is that we attempt to keep Python 2.4 compatibiltiy for those packages). I certainly don't think anybody here would argue against having the big 'compattest' tests get run: even the splitters and whittlers (I might be called one of those) are willing to help fix things when a change to one package breaks the others. If we could get automated testing of the various compatibility sets established, with automated reporting of failures, I think we would see a huge improvement in the signal here: instead of arguing hypotheticals, we would be focused on fixing real breakages. +1 Hopefully the discussions about buildbots will bear fruit. Working on that. :) It would be easier to find leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK). These specific leading developers could report and synchronize with a ZTK release manager, though. There's nothing preventing people from doing this AFAICT. If someone is interested in pursuing a change to a package or collection of packages, they can do so with or without some organizational structure. Problems would arise only if they proposed a backward incompatible change, which isn't to say that backward-incompatible changes are impossible. We still mostly treat them as off-limits, even the three years after we started the effort to break the monolith apart. That change should have made it technically feasible to do backward-incompatible changes, but we haven't yet worked out how to make that happen. I wonder how many situations there are where backward incompatible changes are needed to unblock other work. I don't suppose we have a list of these do we? One thing that makes problems like this really hard is that email is such a terrible tool for much of the needed communication. It would be nice if we could do something more sprinty. I don't want to help if it involves drawn out email discussions, but, FWIW, I'd be willing to allocate some concentrated blocks of time for high-bandwidth discussion and work. Here's a save the date: gocept is organising a summit/sprint close to this year's DZUG conference which will also have sprinting time and space all over. So that's definitely high-bandwidth. :) The summit itself will be more oriented to getting the Zope developer group organies so we can have less of the meta discussions on the mailing list (that would probably be the part Jim is uninterested in ;)). I'd like to keep that focused on a single day though and use the rest of the sprinting time for getting stuff done. So, anyway, here's the dates: 13. September 2010: zope-dev summit in Halle (Saale), Germany 14. September 2010: sprint in Halle (Saale) 15.-17. September 2010: DZUG conference in Dresden, Geramny 18.-19. September 2010: post-conference sprinting days in Dresden I'd love to see as many of you zope-dev guys here in our offices. :) Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) ·
Re: [Zope-dev] Possible DateTime timezone-related regression in Zope 2.12
On 05.03.10 10:53, Andreas Zeidler wrote: also, look at the following (using `DateTime` 2.12): $ ~/plone/coredev/branches/4.0/bin/zopepy from DateTime import DateTime now = DateTime() now == DateTime(now) True now == DateTime(now.ISO()) False this is a _pretty_ behaviour, to say the least! :) heh, that should have read _pretty_ strange, btw... :) so, after some more reading and discussing this with Stefan, it seems the example should really use `ISO8601()` instead of `ISO()`. however, this fails as well: $ ~/plone/coredev/branches/4.0/bin/zopepy from DateTime import DateTime now = DateTime() now == DateTime(now.ISO8601()) False now.ISO8601() '2010-03-05T11:06:09+01:00' unlike stated in `DateTime.interfaces` the string returned `ISO8601` method does not contain the time zone. tres, do you have any comments on this? cheers, andi -- zeidler it consulting - http://zitc.de/ - i...@zitc.de friedelstraße 31 - 12047 berlin - telefon +49 30 25563779 pgp key at http://zitc.de/pgp - http://wwwkeys.de.pgp.net/ plone 4.0 alpha released! -- http://plone.org/products/plone/ ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Possible DateTime timezone-related regression in Zope 2.12
On 05.03.10 11:10, Andreas Zeidler wrote:
now == DateTime(now.ISO8601())
False
now.ISO8601()
'2010-03-05T11:06:09+01:00'
unlike stated in `DateTime.interfaces` the string returned `ISO8601`
method does not contain the time zone.
oops, it does, of course (i think i'm still a bit too tired :)). sorry
about this! :)
the above example fails, because `now` contains the microseconds, but
`ISO8601` doesn't include them:
now - DateTime(now.ISO8601())
1.9525578703703702e-06
now, DateTime(now.ISO8601())
(DateTime('2010/03/05 11:16:9.168701 GMT+1'),
DateTime('2010/03/05 11:16:09 GMT+1'))
tres, do you have any comments on this?
nevermind, i think now it should be clear what needs to be changed... :)
cheers,
andi
--
zeidler it consulting - http://zitc.de/ - i...@zitc.de
friedelstraße 31 - 12047 berlin - telefon +49 30 25563779
pgp key at http://zitc.de/pgp - http://wwwkeys.de.pgp.net/
plone 4.0 alpha released! -- http://plone.org/products/plone/
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Automating tests of the ZTK / zopeapp package sets
On Thu, Mar 04, 2010 at 04:10:29PM -0500, Tres Seaver wrote: After successfully configuring the Hudson[1] continuous integration server yesterday to test the various repoze.* packages[2], I thought I would experiment with using it to drive the compatibility tests for ZTK and zopeapp. Here is what I did: In a shell on my server: $ wget http://hudson-ci.org/latest/hudson.war $ java -jar hudson.war --httpPort=1 As a sysadmin I preferred echo deb http://hudson-ci.org/debian binary/ /etc/apt/sources.list.d/hudson.list sudo apt-get update sudo apt-get install hudson # Visit http://server:8080/ or edit /etc/default/hudson to change the # port number This makes it run in a separate Unix user account, start automatically on system boot, and it lets me get updates easily. In the browser, visit 'http://laguna.palladion.com:1, and configure the server: - Enable security - allow signup - using Matrix - set up 'tseaver' as having all permissions - set up 'anonymous' as having 'Read' permission overall and for jobs - Register as 'tseaver' - Turn off signup (no, I don't want you guys running arbitrary shell scripts on my server ;). We have commit rights to the SVN repository. We already can run arbitrary shell scripts on your server. ;) Of course the whole audit trail thing might make it less attractive ;) - Create a job, 'ztk-trunk-py26': ... - Added build step as a shell script:: cd trunk /opt/Python-2.6.4/bin/python bootstrap.py bin/buildout bin/test-ztk bin/test-zopeapp - Ran a build manually. The first one barfed due to a typo in my script, but the second one ran, taking 21 minutes. And the third and subsequent ones took 11 minutes. The initial buildout run is not fast. The test runs successfully to completion, but doesn't produce statistics in a form that Hudson understands. The following changes to 'zope.testing.testrunner' would make the output more informative: - Add a flag like the '--with-xunit' flag to the Nose testrunner, which dumped results into a JUnit-compatible XML file[3]. I'd like that. - Add a flag like the '--with-xcompat' flag to the Nose testrunner (when run with the 'nosexcover' plugin), producing a Cobertura- compabtible XML file[4]. I believe you can do that with Ned Batchelder's coverage.py if you include 'coverage' in your eggs list in buildout.cfg and run the tests as bin/coverage run bin/test [options as usual] bin/coverage xml Downside: Hudson runs the steps as a shell script with 'set -e' enabled, which means that the first command that fails stops the script. You won't get coverage results if your tests fail. (Also, if ztk tests fail, zopeapp tests won't even be run). The syntax for combining coverage files produced by several test runs is left as an exercise for the reader: http://nedbatchelder.com/code/coverage/cmd.html At that point, the build process would be nicely automated, with browsable results including coverage, pretty graphs, etc. See the 'repoze' jobs for examples of those outputs. Nice. Incidentally, I recommend the green balls plugin that I see you're already using for hudson.repoze.org. Blue doesn't really signal job was successful to me. Questions for discussion: - I find the prettier interface and easier setup than buildbot worth running a 3rd-party Java app (rather than a 3rd party Python app). Would that be acceptable among the folks running our automated tests? A year ago I'd have said there will be no PHP or Java on my servers. A month ago I installed Hudson because Buildbot finally got to me. I'm still holding the fort against PHP, though ;) - Is it worth adding the XML support for test results and coverage to 'zope.testing.testrunner'? I think so, at least for the test results. - Would people be willing to run Hudson permanently on enough hosts to make this a reasonable replacement for buildbot (we need Windows, too)? I'd have to overcome my paranoia of letting somebody else define what shell commands get run on my servers, but it seems enough people are already providing Linux build slaves. - Hudson seems general purpose enough to allow automating other code- related tasks, e.g., production of dependency graphs. Maybe we could even automate building Windows installers? It would be *awesome* if people making releases of core packages like zope.interface didn't need to build Windows binary eggs by themselves. I will try to leave the ZTK Hudson server up for a little bit, to allow folks to see what it can deliver already. [1] http://hudson-ci.org/ [2] http://hudson.repoze.org/ [3] http://old.nabble.com/schema-for-junit-xml-output-td22193385.html [4] http://cobertura.sourceforge.net/xml/coverage-01.dtd Regards, Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature
[Zope-dev] Zope Tests: 6 OK
Summary of messages to the zope-tests list. Period Thu Mar 4 12:00:00 2010 UTC to Fri Mar 5 12:00:00 2010 UTC. There were 6 messages: 6 from Zope Tests. Tests passed OK --- Subject: OK : Zope-2.10 Python-2.4.6 : Linux From: Zope Tests Date: Thu Mar 4 20:37:17 EST 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-March/013677.html Subject: OK : Zope-2.11 Python-2.4.6 : Linux From: Zope Tests Date: Thu Mar 4 20:39:17 EST 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-March/013678.html Subject: OK : Zope-2.12 Python-2.6.4 : Linux From: Zope Tests Date: Thu Mar 4 20:41:17 EST 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-March/013679.html Subject: OK : Zope-2.12-alltests Python-2.6.4 : Linux From: Zope Tests Date: Thu Mar 4 20:43:17 EST 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-March/013680.html Subject: OK : Zope-trunk Python-2.6.4 : Linux From: Zope Tests Date: Thu Mar 4 20:45:17 EST 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-March/013681.html Subject: OK : Zope-trunk-alltests Python-2.6.4 : Linux From: Zope Tests Date: Thu Mar 4 20:47:17 EST 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-March/013682.html ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZTK compatibility testing
Hi, as promised I'm currently trying to write down the instructions for running ZTK tests. I need some feedback on our process. I figured there are various scenarios for which we have tools: 1. Ensure that a branch of the ZTK stays in working order For that the ZTK has test runners created by z3c.recipe.compattest which ensures that the individual tests of the ZTK packages with their respective dependencies using the specific ZTK versions are ok. This helps when evolving the versions in the ZTK. Those should be run nightly as is done by some of the buildbots. 2. Ensure that a change of a package doesn't break the set 2a: ensuring compatibility with a specific branch of the set The developer working on package X can use a compat test runner and a specific ZTK buildout configuration to run the other packages' tests combined with the development version of package X. This can/should be done by individual developers for the (newest/olders?) ZTK version they target their change for. If we can encode the target ZTK version for a specific package in that packages' configuration then we can also have nightly builds that perform this verification automatically. I'm not sure how to encode that, though, because the relation between the package and the ZTK is usually defined the other way around. 2b: ensuring ongoing compatibility of the trunks In addition I wonder how the various trunk versions of ZTK packages should relate to each other. Should they always build? Should we accept breakage of all trunks with each other or not? Also, there is not version of the ZTK (I think) that specifies 'use the trunk of everything', right? Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZTK compatibility testing
On Fri, Mar 5, 2010 at 8:56 AM, Christian Theune c...@gocept.com wrote: ... 2b: ensuring ongoing compatibility of the trunks In addition I wonder how the various trunk versions of ZTK packages should relate to each other. Should they always build? Should we accept breakage of all trunks with each other or not? Also, there is not version of the ZTK (I think) that specifies 'use the trunk of everything', right? IMO, there should rarely, if ever, be changes on the trunk that aren't in the most recent known good ZTK distribution. The way I've done this is: - Check out the ZTK - Check out a working copy of the package(s) of interest and build as develop eggs with the rest of the ZTK. - Hack till I'm happy and the ZTK tests pass. - Check in changes to the packages and make new releases. - Update the ZTK buildout to use the new releases and test again. - Test with multiple Python versions. - Check in changes to a branch of the ZTK. - Run tests of the branch on windows. If they pass, merge the ZTK branch to trunk. With buildbouts, ideally there'd be a stage branch we could merge to, wait for the buildbots to to their thing and then merge to trunk when tests pass on all platforms. I really think we want to avoid long-lasting unreleased trunk changes to or long-lasting newer releases not in the KGS for ZTK packages. Jim -- Jim Fulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Possible DateTime timezone-related regression in Zope 2.12
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Martin Aspeli wrote:
Hi,
We have a failing test in plone.app.dexterity 1.0a7. This is simply
trying to compare two dates:
from DateTime import DateTime
DateTime() DateTime(md.CreationDate())
True
At least here in Australia, the second test fails. Right now, the
following expressions are:
DateTime(): DateTime('2010/01/10 11:20:24.718203 GMT+8')
md.CreationDate(): '2010-01-10 11:19:57'
DateTime(md.CreationDate()): DateTime('2010/01/10 11:19:57 GMT+0')
On Zope 2.10, it's a different story:
DateTime(): DateTime('2010/01/10 11:34:01.508 GMT+8')
md.CreationDate(): '2010-01-10 11:24:42'
DateTime(md.CreationDate()): DateTime('2010/01/10 11:24:42 GMT+8')
Andi Zeidler looked into it briefly, and said the following:
imho, this is due a bug in `DateTime` 2.12.0. the newer version behaves
differently when it's initialized via a string representation of a date
— it interprets the given date to be GMT while before it was taken to be
from your local time zone:
$ cd ~/plone/coredev/branches
$ cat foo.py
from DateTime import DateTime
print DateTime('2010-01-09 00:34:37')
$ 3.3/bin/zopepy foo.py
2010/01/09 00:34:37 GMT+1
$ 4.0/bin/zopepy foo.py
2010/01/09 00:34:37 GMT+0
so in your test, `DateTime(md.CreationDate())` will always be the
current time, but with an implicitly added 'GMT+0' while `DateTime()`
will be the current time in your local time zone. so if i'm not
mistaken, on plone 4.0 the test with fail for you an me (in 'GMT+x' time
zones) and pass in the u.s. fun! :)
Does anyone know if this change was deliberate, or what may have happened?
Why would you be parsing the 'CreationDate' return value (a string)
instead of just using the 'created' value (already a DateTime)?
Tres.
- --
===
Tres Seaver +1 540-429-0999 tsea...@palladion.com
Palladion Software Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkuRU9sACgkQ+gerLs4ltQ4ItQCgpgfgzo9SKBjx4cXVxPnps4h6
8RAAoKrV/Z2K+WLPBzuWd+XhZHlA7pRW
=CryU
-END PGP SIGNATURE-
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Uses of the ZTK and how it relates to management
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Christian Theune wrote: On 03/04/2010 04:13 PM, Jim Fulton wrote: On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella kob...@kobold.it wrote: * 2010-03-03 21:44, Jim Fulton wrote: The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK. It would be great if that were true. That's been my understanding all the time. I think we currently don't do it because it might be non-obvious to the individual developers at the time of check-in and due to underused tools. We may not require that a single developer run all test suites in all combinations we desire to be compatible, but with the ongoing effort to improve the nightly/automated builds we should get into better shape there. The one issue with delayed testing is that we would have to impose a rule which requires cool down time after a checkin before making a release. That's probably not a terrible thing anyway. I would think that requiring the buildbots to clear before tagging a new release of the ztk.cfg file is a good cool down, but we can't require that people not release the included packages: the ZTK isn't really testable without having them released (a chicken-and-egg problem). I would say that updating ztk.cfg to new major versions of sub-packages (ones with new features, or potential backward compatibility issues) requires a great deal more care than updating it to use a new bugfix only release of a package already in the ZTK. Adding or removing packages from the ZTK should probably also be a branch only item, with discussion required here before merging. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuRVaYACgkQ+gerLs4ltQ5J3gCg0tFeV6g1lxDk9NdxKk9Ze1Is ceIAoIJjcsL/Ya7Ei4H5lBxqE+sdGDsh =epLA -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] SVN: DateTime/trunk/src/DateTime/tests/testDateTime.py add a failing test for a regression in parsing ISO format datetimes from DateTime 2.10, as discussed at http://dev.plone.org/plone
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
David Glick wrote:
Log message for revision 109667:
add a failing test for a regression in parsing ISO format datetimes from
DateTime 2.10, as discussed at http://dev.plone.org/plone/ticket/10140
...note that this will give a false positive if run on a computer where GMT
is the local timezone
Please don't deliberately check in failing tests on the trunk. If you
need to do this, make a branch, and ask on the mailing list for people
to investigate your branch.
Changed:
U DateTime/trunk/src/DateTime/tests/testDateTime.py
-=-
Modified: DateTime/trunk/src/DateTime/tests/testDateTime.py
===
--- DateTime/trunk/src/DateTime/tests/testDateTime.py 2010-03-05 03:41:57 UTC
(rev 109666)
+++ DateTime/trunk/src/DateTime/tests/testDateTime.py 2010-03-05 07:04:12 UTC
(rev 109667)
@@ -332,6 +332,7 @@
ref2 = DateTime('2006/11/6 10:30 GMT')
ref3 = DateTime('2004/06/14 14:30:15 GMT-3')
ref4 = DateTime('2006/01/01 GMT')
+ref5 = DateTime('2006/01/01')
Note that this a nonreproducible test value: parsing the string with
slashes but no timezone always gives you local time.
Tres.
- --
===
Tres Seaver +1 540-429-0999 tsea...@palladion.com
Palladion Software Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkuRWQQACgkQ+gerLs4ltQ7fUwCg1sqO48X6vjW/pgyq+2I4OGV0
j/IAn0/IZy+/d9+O/nEykkTLBLVYxdua
=paH6
-END PGP SIGNATURE-
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] SVN: DateTime/trunk/src/DateTime/tests/testDateTime.py add a failing test for a regression in parsing ISO format datetimes from DateTime 2.10, as discussed at http://dev.plone.org/plone
Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David Glick wrote: Log message for revision 109667: add a failing test for a regression in parsing ISO format datetimes from DateTime 2.10, as discussed at http://dev.plone.org/plone/ticket/10140 ...note that this will give a false positive if run on a computer where GMT is the local timezone Please don't deliberately check in failing tests on the trunk. If you need to do this, make a branch, and ask on the mailing list for people to investigate your branch. Why not? Trunk is (well, was) broken. This makes it clear. The regression actually happened ages ago, but no-one had written a decent test for the original functionality, so the regression was only discovered in application software. This test corrects that omission, and helped a few people co-ordinate addressing the issue. Why benefit do we get from not making the breakage explicit to everyone? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
