Re: [Evolution-hackers] nightly builds of Evolution +testing withSyncEvolution
On Wed, 2008-02-13 at 09:44 +0530, Srinivasa Ragavan wrote: > On Tue, 2008-02-12 at 19:55 +0100, Patrick Ohly wrote: > > I can run "make check" after a build. Are all tests going to be run by > > that? > I doubt, if that is kept updated to source. Then how do I run the existing tests instead? -- Bye, Patrick Ohly -- [EMAIL PROTECTED] http://www.estamos.de/ ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] nightly builds of Evolution +testing withSyncEvolution
On Tue, 2008-02-12 at 19:55 +0100, Patrick Ohly wrote: > On Mon, 2008-02-11 at 15:32 +0530, Srinivasa Ragavan wrote: > > On Sun, 2008-02-10 at 12:18 +0100, Patrick Ohly wrote: > > > I wonder whether this regular building and testing is of interest to > > > anybody else? The build script sends out a short summary email which > > > links to full logs for each night; I could easily add other recipients. > > > For the Evolution build these include the changes since the last build. > > > > > I think it may prove very helpful in the long run, where some commits > > cause some side effects, which we may not easily identify in normal > > cases. But the test script gotto be foolproof to make this really > > useful. > > They have worked fine so far. I'll see how it'll go in the future. > > > > Are there other tests of the EDS API which I should run? If Novel > > > already does regular testing this probably isn't needed. > > AFAIK, we don't have much automated tests except the tests under the > > respective folders. Akhil, correct me if I'm wrong. Patrick, if your > > test script could cover some cases from those test files also, it will > > be really useful. > > I can run "make check" after a build. Are all tests going to be run by > that? I doubt, if that is kept updated to source. -Srini. ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] nightly builds of Evolution + testing withSyncEvolution
On Mon, 2008-02-11 at 15:32 +0530, Srinivasa Ragavan wrote: > On Sun, 2008-02-10 at 12:18 +0100, Patrick Ohly wrote: > > I wonder whether this regular building and testing is of interest to > > anybody else? The build script sends out a short summary email which > > links to full logs for each night; I could easily add other recipients. > > For the Evolution build these include the changes since the last build. > > > I think it may prove very helpful in the long run, where some commits > cause some side effects, which we may not easily identify in normal > cases. But the test script gotto be foolproof to make this really > useful. They have worked fine so far. I'll see how it'll go in the future. > > Are there other tests of the EDS API which I should run? If Novel > > already does regular testing this probably isn't needed. > AFAIK, we don't have much automated tests except the tests under the > respective folders. Akhil, correct me if I'm wrong. Patrick, if your > test script could cover some cases from those test files also, it will > be really useful. I can run "make check" after a build. Are all tests going to be run by that? How do I detect failures? I just tried it on evolution-data-server and got e.g.: [...] Failed 48/662 tests, 92% okay Failed tests: 2/23 2/24 2/26 2/27 2/28 2/29 2/32 2/47 2/48 2/50 2/51 2/52 2/53 2/56 2/71 2/72 2/74 2/75 2/76 2/77 2/80 2/95 2/96 2/98 2/99 2/100 2/101 2/104 2/119 2/120 2/122 2/123 2/124 2/125 10/295 10/325 14/344 14/345 14/346 15/347 15/349 15/350 15/351 27/486 31/503 32/505 39/599 39/600 PASS: regression == All 1 tests passed == make[5]: Leaving directory `/tmp/runtests/head/tmp/evolutiontrunk-build/evolution-data-server/calendar/libical/src/test' The "make check" at top level returned no error code. Looking at calendar/libical/src/test/regression.c I see that it always returns 0, so I guess I have to scan the log output to find failures, right? Are the failures above something that should be reported? To make regular testing useful failing tests either have to be skipped or fixed in a timely fashion. -- Bye, Patrick Ohly -- [EMAIL PROTECTED] http://www.estamos.de/ ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] nightly builds of Evolution + testing withSyncEvolution
Hey Patrick, On Sun, 2008-02-10 at 12:18 +0100, Patrick Ohly wrote: > Hello, > > I finally got all pieces together (in particular most of the recent > GNOME libs which are missing in Debian Etch), so now I'm building > Evolution trunk each night using Paul Smith's most excellent Makefile > [1]. > Cool. > After each build I then run SyncEvolution tests against the EDS API. > These tests cover the synchronous libebook (contacts) and libecal > (calendar, todos, memos). My primary motivation is to catch changes > affecting SyncEvolution before the release, not after it ;-} Sounds really nice to me. > > I wonder whether this regular building and testing is of interest to > anybody else? The build script sends out a short summary email which > links to full logs for each night; I could easily add other recipients. > For the Evolution build these include the changes since the last build. > I think it may prove very helpful in the long run, where some commits cause some side effects, which we may not easily identify in normal cases. But the test script gotto be foolproof to make this really useful. > I'll probably won't be able to check the logs each day, but if I do and > find build problems, how should I report them? As entry in the GNOME > Bugzilla or an email to this list? Feel free to file bugs on it. > > Are there other tests of the EDS API which I should run? If Novel > already does regular testing this probably isn't needed. AFAIK, we don't have much automated tests except the tests under the respective folders. Akhil, correct me if I'm wrong. Patrick, if your test script could cover some cases from those test files also, it will be really useful. > > [1] http://mad-scientist.us/evolution.html This is a very nice initiative IMO. Thanks for the trigger. -Srini. ___ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers
