On 4/11/06, Mark Hindess wrote: > Personally, obviously, I'd expect people to run the tests before > committing. > > However, I notice that since enabling the security tests - which fork > for every test - that the tests take over half an hour to run now on > our Linux build machine. So I can see why enthusiasm might lead to > people not running all the tests but instead perhaps just running > those from the module you are changing.
I think that this message[1] relates to the problem with running big test suites. So should we do the similar thing i.e. define which test suites should be run for each module? Thanks, Stepan. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/[EMAIL PROTECTED] > George just commented in a commit message backing out some changes in > text that all the text tests passed even though the changed caused > failures elsewhere. > > Regards, > Mark. > > On 4/11/06, Geir Magnusson Jr <[EMAIL PROTECTED]> wrote: > > Just curious (and this isn't a criticism - I'm just as guilty of not > > doing this)... > > > > Don't you run the tests before committing? > > > > geir > > > > George Harley wrote: > > > Hi, > > > > > > It *seems* like things started failing after I committed the changes > for > > > HARMONY-205 last night. I'm looking into this now. If the > investigation > > > begins to take up too much time I will back the changes out. > > > > > > Best regards, > > > George > > > > > > > > > Stepan Mishura wrote: > > >> The same for me + DatagramChannelTest > > >> > > >> Thanks, > > >> Stepan. > > >> > > >> On 4/11/06, Mark Hindess wrote: > > >> > > >>> No. These: > > >>> > > >>> F > > >>> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder > > >>> > > >>> E > > >>> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01 > > >>> > > >>> E > > >>> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02 > > >>> > > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt > > >>> E > > >>> > org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode > > >>> > > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid > > >>> F java.security.cert.X509CertSelectorTest.testMatch > > >>> F java.security.cert.X509CertSelectorTest.testClone > > >>> F tests.api.java.sql.DateTest.testSetTimelong > > >>> F tests.api.java.sql.DateTest.testToString > > >>> F tests.api.java.sql.DateTest.testValueOf > > >>> F tests.api.java.sql.TimestampTest.testSetNanosint > > >>> F tests.api.java.sql.TimestampTest.testToString > > >>> F tests.api.java.util.DateTest.test_toGMTString > > >>> F tests.api.java.util.DateTest.test_toString > > >>> > > >>> Regards, > > >>> Mark. > > >>> > > >>> On 4/11/06, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > >>> > > >>>> On 4/11/06, Mark Hindess wrote: > > >>>> > > >>>>> Yes. I was using the failureproperty mechanism. Trying to get > this > > >>>>> property propogated back to the top level ant file was what I was > > >>>>> having trouble with. > > >>>>> > > >>>>> Using a file as you suggest might help. I'll give that a try > > >>>>> > > >>> shortly... > > >>> > > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111. > > >>>>> > > >>>> I guess that you have 9 tests from DatagramChannelTest passed. And > > >>>> 12 + > > >>>> > > >>> 3 = > > >>> > > >>>> 15 :-) > > >>>> > > >>>> (And > > >>>> > > >>>>> there are typos - "mathc" should be "match" - in the failure > messages > > >>>>> for java.security.cert testMatch and testClone.) > > >>>>> > > >>>> I've fixed typo. > > >>>> > > >>>> Thanks, > > >>>> Stepan. > > >>>> > > >>>> Regards, > > >>>> > > >>>>> Mark. > > >>>>> > > >>>>> On 4/11/06, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > >>>>> > > >>>>>> On 4/11/06, Mark Hindess wrote: > > >>>>>> > > >>>>>> > > >>>>>>> Stepan, > > >>>>>>> > > >>>>>>> I have another build running (but without notifications going to > > >>>>>>> > > >>> the > > >>> > > >>>>>>> list) that runs: > > >>>>>>> > > >>>>>>> 1) build (with reference jdk) > > >>>>>>> 2) build with what we created with 1) > > >>>>>>> 3) test > > >>>>>>> 4) create classlists and compare with class load data for > > >>>>>>> > > >>> applications > > >>> > > >>>>>>> (tomcat, geronimo, continuum, etc.) > > >>>>>>> 5) generate JAPI results > > >>>>>>> > > >>>>>>> I'd like to fail this build at step 3, but I can't see an easy > way > > >>>>>>> > > >>> to > > >>> > > >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail > if > > >>>>>>> > > >>> any > > >>> > > >>>>>>> of the module test sub-targets had test failures. I could parse > > >>>>>>> > > >>> the > > >>> > > >>>>>>> output I suppose, but I'd rather get ant to propagate the junit > > >>>>>>> > > >>> tasks > > >>> > > >>>>>>> failure property back up to the top level. I've tried a couple > of > > >>>>>>> things and none seem to work. Any suggestions welcome. > > >>>>>>> > > >>>>>> Mark, did you try failureproperty parameter for junit task? > > >>>>>> We may add it to ant sub-targets to raise a flag, for example, > > >>>>>> > > >>> create > > >>> > > >>>>> file " > > >>>>> > > >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And > in > > >>>>>> > > >>> the > > >>> > > >>>>> end > > >>>>> > > >>>>>> of tests suite run check whether this file exists on not. > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Stepan. > > >>>>>> > > >>>>>> > > >>>>>> Regards, > > >>>>>> > > >>>>>>> Mark. > > >>>>>>> > > >>>>>>> On 4/11/06, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > >>>>>>> > > >>>>>>>> Hi, > > >>>>>>>> > > >>>>>>>> I've checked out at morning last updates, built the code base > > >>>>>>>> > > >>> and > > >>> > > >>>>> run > > >>>>> > > >>>>>>> the > > >>>>>>> > > >>>>>>>> tests …and there are 24 tests failures! > > >>>>>>>> > > >>>>>>>> There are 9 tests failures in > > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest– > > >>>>>>>> > > >>> I > > >>> > > >>>>> saw > > >>>>> > > >>>>>>> these > > >>>>>>> > > >>>>>>>> failures before from time to time. It seems that tests depend > on > > >>>>>>>> > > >>>>> some > > >>>>> > > >>>>>>> race > > >>>>>>> > > >>>>>>>> conditions because they may pass if I rerun them. Paulex, are > > >>>>>>>> > > >>> these > > >>> > > >>>>>>> tests > > >>>>>>> > > >>>>>>>> passing for you? > > >>>>>>>> > > >>>>>>>> And there are new 15 test failures. So now if I'll make a code > > >>>>>>>> > > >>>>> update > > >>>>> > > >>>>>>> or a > > >>>>>>> > > >>>>>>>> bug fix how I can be 100% sure that I don't do any regression? > > >>>>>>>> > > >>>>>>>> Currently if a commit breaks the Harmony classlib build a > > >>>>>>>> > > >>>>> notification > > >>>>> > > >>>>>>> with > > >>>>>>> > > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will > > >>>>>>>> > > >>> be > > >>> > > >>>>> send > > >>>>> > > >>>>>>> to > > >>>>>>> > > >>>>>>>> harmony-commits mailing list. Is it possible to have the same > > >>>>>>>> > > >>> for > > >>> > > >>>>> tests? > > >>>>> > > >>>>>>> I > > >>>>>>> > > >>>>>>>> mean that after completing automatic build the existing > classlib > > >>>>>>>> > > >>>>> tests > > >>>>> > > >>>>>>> suite > > >>>>>>> > > >>>>>>>> should be run. If there are failing tests then a report > > >>>>>>>> > > >>> notification > > >>> > > >>>>>>> with > > >>>>>>> > > >>>>>>>> corresponding subject will be send. > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> Stepan Mishura > > >>>>>>>> Intel Middleware Products Division > > >>>>>>>> > > >>>>>>> -- > > >>>>>>> Mark Hindess <[EMAIL PROTECTED]> > > >>>>>>> IBM Java Technology Centre, UK. > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>> > --------------------------------------------------------------------- > > >>> > > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>>>>> To unsubscribe, e-mail: > > >>>>>>> > > >>> [EMAIL PROTECTED] > > >>> > > >>>>>>> For additional commands, e-mail: > > >>>>>>> > > >>> [EMAIL PROTECTED] > > >>> > > >>>>>>> > > >>>>>> -- > > >>>>>> > > >>>>>> > > >>> > --------------------------------------------------------------------- > > >>> > > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>>>> To unsubscribe, e-mail: > [EMAIL PROTECTED] > > >>>>>> For additional commands, e-mail: > > >>>>>> > > >>> [EMAIL PROTECTED] > > >>> > > >>>>>> Thanks, > > >>>>>> Stepan Mishura > > >>>>>> Intel Middleware Products Division > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>> -- > > >>>>> Mark Hindess <[EMAIL PROTECTED]> > > >>>>> IBM Java Technology Centre, UK. > > >>>>> > > >>>>> > --------------------------------------------------------------------- > > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>>> To unsubscribe, e-mail: > [EMAIL PROTECTED] > > >>>>> For additional commands, e-mail: > [EMAIL PROTECTED] > > >>>>> > > >>>>> > > >>>>> > > >>>> -- > > >>>> > --------------------------------------------------------------------- > > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>> To unsubscribe, e-mail: > [EMAIL PROTECTED] > > >>>> For additional commands, e-mail: > [EMAIL PROTECTED] > > >>>> > > >>>> Thanks, > > >>>> Stepan Mishura > > >>>> Intel Middleware Products Division > > >>>> > > >>>> > > >>>> > > >>> -- > > >>> Mark Hindess <[EMAIL PROTECTED]> > > >>> IBM Java Technology Centre, UK. > > >>> > > >>> > --------------------------------------------------------------------- > > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >>> For additional commands, e-mail: > [EMAIL PROTECTED] > > >>> > > >>> > > >>> > > >> > > >> > > >> -- > > >> --------------------------------------------------------------------- > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >> For additional commands, e-mail: > [EMAIL PROTECTED] > > >> > > >> Thanks, > > >> Stepan Mishura > > >> Intel Middleware Products Division > > >> > > >> > > > > > > > > > --------------------------------------------------------------------- > > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > -- > Mark Hindess <[EMAIL PROTECTED]> > IBM Java Technology Centre, UK. > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Thanks, Stepan Mishura Intel Middleware Products Division
