For the record, -Dmaven.test.failure.ignore seems to do the right thing, and the JDK7 build now only has 7 test failures (+ 4 ignored):
http://ci.infinispan.org/viewLog.html?buildId=5912&tab=buildResultsDiv&buildTypeId=bt8 On Tue, Feb 4, 2014 at 4:36 PM, Galder Zamarreño <gal...@redhat.com> wrote: > All, > > Sanne, Pedro, Dan and I had a very productive discussion on IRC on this > topic [1]. > > We've decided that instead of disabling tests, we need them to run in > order to get recent stacktraces, logs, etc. So, we've decided to create a > new test group called "unstable". This test group would only be run in CI > once a day and it'd be run in a different build. This build would also > enable TRACE logging for standalone and server tests. For server, I need to > create a task to do this selectively. > > The rest of builds, masters and PRS would not run the "unstable" group, > and would not have TRACE enabled. > > The responsibility of unstable tests are the component owners. They need > to handle them and decide what to do with them. > > Cheers, > > [1] https://gist.github.com/galderz/3563d1b23b5d50f80d82 > > On 04 Feb 2014, at 15:10, Galder Zamarreño <gal...@redhat.com> wrote: > > > > > On 04 Feb 2014, at 14:47, Dan Berindei <dan.berin...@gmail.com> wrote: > > > >> > >> > >> > >> On Tue, Feb 4, 2014 at 3:03 PM, Galder Zamarreño <gal...@redhat.com> > wrote: > >> > >> On 04 Feb 2014, at 13:50, Dan Berindei <dan.berin...@gmail.com> wrote: > >> > >>> > >>> On Tue, Feb 4, 2014 at 2:36 PM, Galder Zamarreño <gal...@redhat.com> > wrote: > >>> Narrowing down the list now, since this is a problem of how our CI is > doing builds. > >>> > >>> These logs are retrieved from [1]. > >>> > >>> Dunno how our CI is configured but this is odd. Seems like the build > is halt due to test failures, but it continues somehow? I mean, the jars > are not being produced properly, but the build is not halting. > >>> > >>> We run the build with -fn (fail-never), so the build should never be > halted because of a test failure. The configuration is here: > http://ci.infinispan.org/admin/editRunType.html?id=buildType:bt8&runnerId=RUNNER_1 > >> > >> ^ That's not working as expected, see the build log, my snippets...etc. > >> > >> Sorry, I didn't understand what's happening in those snippets. > > > > The log shows it quite clearly that after those tests fail, nothing else > runs in that module, including producing the jar. It halts. That's > > > >> All I saw was an Ant script that doesn't do what it's supposed to do :) > > > > The ant script not doing it's job is because there modules are not > completing the build. There's a direct correlation between the three > modules that fail with tests and the 3 modules that are copying an empty > folder instead of the jar. > > > >> > >> I did see some differences in the configuration between the JDK6 and > the JDK7 builds: > >> * JDK7 uses -fn and JDK6 uses -Dmaven.test.failure.ignore > >> * JDK7 uses -nsu (no snapshot updates), JDK6 doesn't > >> > >> I've changed both builds to use -Dmaven.test.failure.ignore and -nsu, > let's see how it goes. > > > > You are solving the wrong problem. > > > >> > >> > >> > >>> > >>> > >>> > >>> It's about time we did the following: > >>> 1) Any test failures should halt the build there and then. IOW, do not > continue the build at all. > >>> > >>> Will having 100 tests in one run and 2000 tests in another really help? > >> > >> As you disable randomly failing tests, and do not integrate commits > making the testsuite fail, these number should even out. > >> > >> Not integrating commits that fail every time is easy, not integrating > commits that fail randomly (maybe only in some environments) is trickier. > > > > I know it's tricky, but the only thing we can do is disable those > really. I don't see how keeping them enabled is helping at all. > > > >> > >> > >>> > >>> > >>> 2) Any tests that fail randomly should be disabled. > >>> > >>> Let's go ahead and disable all the server tests then? ;) > >> > >> Those server tests that are randomly failing should be disabled and > looked at. Those tests that are failing as a result of container not > starting are side effects of things not working properly, and these should > not be disabled. > >> > >> Why treat the tests that are failing because of a build problem > differently? What about the tests that fail only on IBM JDK6? > > > > Disable and indicate that the test fails on IBM JDK6. Once the issue is > fixed, reenable it. > > > >> > >> > >>> > >>> > >>> > >>> Cheers, > >>> > >>> [1] https://dl.dropboxusercontent.com/u/6148072/does-not-work.log > >>> > >>> On 04 Feb 2014, at 13:30, Galder Zamarreño <gal...@redhat.com> wrote: > >>> > >>>> > >>>> On 04 Feb 2014, at 10:38, Stuart Douglas <stuart.w.doug...@gmail.com> > wrote: > >>>> > >>>>> It is almost certainly something to do with this: > >>>>> > >>>>> <module-def name="org.infinispan.server.rest" > src="${infinispan.server.modules.dir}"> > >>>>> > >>>>> <maven-resource-with-classifier group="org.infinispan" > artifact="infinispan-server-rest" classifier="classes" /> > >>>>> > >>>>> </module-def> > >>>>> > >>>>> I guess sometimes the classes artefact is being attached as a > reference to the classes directory, rather than a reference to a jar, which > causes the issue. > >>>> > >>>> Here's a gist with a subset of the build log [1]. When it works fine, > it's copying a jar, when it's not, it's copying an empty folder. > >>>> > >>>> However, this is not only happening for the > org.infinispan.server.rest module, others show the same issue [2]. What > seems to be a pattern is that it only happens with modules that are built > by us, it's not happening for modules coming with the base AS/WF instance. > >>>> > >>>> I've traced back and this might be due to build failures that are not > producing the right jars [3]. > >>>> > >>>> @Stuart, this is really our problem. Sorry for the inconvenience! > >>>> > >>>> [1] https://gist.github.com/galderz/b9286f385aad1316df51 > >>>> [2] https://gist.github.com/galderz/9e6a9bd9b18b805db323 > >>>> [3] https://gist.github.com/galderz/6ab662a1027cd96cbd8c > >>>> > >>>>> > >>>>> Stuart > >>>>> > >>>>> > >>>>> On Tue, Feb 4, 2014 at 11:14 AM, Galder Zamarreño <gal...@redhat.com> > wrote: > >>>>> > >>>>> On 04 Feb 2014, at 10:01, Stuart Douglas <stuart.w.doug...@gmail.com> > wrote: > >>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Tue, Feb 4, 2014 at 11:00 AM, Stuart Douglas < > stuart.w.doug...@gmail.com> wrote: > >>>>>> Yes, there is nothing in the server code that modified the modules > directory. > >>>>>> > >>>>>> Well, except for the new patching stuff, but that is not really > relevant here. > >>>>> > >>>>> The testsuite AS/WF builds are built out of the distribution build, > which shows the same problem. The distribution we build uses the scripts we > got from AS [1]. > >>>>> > >>>>> Do you see anything in there that could be causing this? We are > using maven-antrun-plugin version 1.3, and take into account the lib.xml in > [2]. > >>>>> > >>>>> Finally, do you have any suggestions on changes we could make to > these files to further debug the issue? > >>>>> > >>>>> Thanks a lot for your help! > >>>>> > >>>>> [1] > https://github.com/infinispan/infinispan/blob/master/server/integration/build/build.xml > >>>>> [2] > https://github.com/infinispan/infinispan/blob/master/server/integration/build/lib.xml > >>>>> > >>>>>> > >>>>>> Stuart > >>>>>> > >>>>>> > >>>>>> Stuart > >>>>>> > >>>>>> > >>>>>> On Tue, Feb 4, 2014 at 10:56 AM, Galder Zamarreño < > gal...@redhat.com> wrote: > >>>>>> > >>>>>> On 04 Feb 2014, at 09:37, Stuart Douglas < > stuart.w.doug...@gmail.com> wrote: > >>>>>> > >>>>>>> This looks like an issue with your environment. The modules > directory is static. Wildfly does not contain any code that messes with it. > I would say the culprit is probably something in either your build process > or your test suite. > >>>>>> > >>>>>> Correction, this is happening with AS 7.2.0.Final (Wildfly 8 used > somewhere else). I guess your answer still applies? > >>>>>> > >>>>>> Cheers, > >>>>>> > >>>>>>> > >>>>>>> Stuart > >>>>>>> > >>>>>>> > >>>>>>> On Tue, Feb 4, 2014 at 10:21 AM, Galder Zamarreño < > gal...@redhat.com> wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> We're having issues with our Infinispan Server integration tests, > which run within Wildfly 8.0.0.Beta1 (as I'm typing I'm wondering if we > should just upgrade it to see if this goes away...?). > >>>>>>> > >>>>>>> Quite often some of the runs fail with error message [1]. > >>>>>>> > >>>>>>> Having looked at the build environment when a run fails, you see > this: > >>>>>>> > >>>>>>> -- > >>>>>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main > >>>>>>> drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<-- a > directory??) > >>>>>>> -rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index > >>>>>>> -rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml > >>>>>>> > >>>>>>> $ ls > modules/system/layers/base/org/infinispan/server/rest/main/classes > >>>>>>> drwxrwxr-x 2 g staff 68B Feb 3 18:41 . > >>>>>>> drwxrwxr-x 5 g staff 170B Feb 3 18:41 .. > >>>>>>> > >>>>>>> $ more > modules/system/layers/base/org/infinispan/server/rest/main/module.xml > >>>>>>> <module xmlns="urn:jboss:module:1.1" > name="org.infinispan.server.rest"> > >>>>>>> ... > >>>>>>> <resource-root path="classes"/> > >>>>>>> ... > >>>>>>> > >>>>>>> This is completely different to what happens with a successful run: > >>>>>>> > >>>>>>> -- > >>>>>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main > >>>>>>> -rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar > (<-- a jar file!) > >>>>>>> -rw-r--r-- 1 g staff 278B Feb 3 19:40 > infinispan-classes.jar.index > >>>>>>> -rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml > >>>>>>> > >>>>>>> $ jar tf > modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar > | grep ExtendedHeaders > >>>>>>> org/infinispan/rest/configuration/ExtendedHeaders.class > >>>>>>> > >>>>>>> $ more > modules/system/layers/base/org/infinispan/server/rest/main/module.xml > >>>>>>> <module xmlns="urn:jboss:module:1.1" > name="org.infinispan.server.rest"> > >>>>>>> ... > >>>>>>> <resource-root path="infinispan-classes.jar"/> > >>>>>>> -- > >>>>>>> > >>>>>>> Anyone can explain what is going on here? Does it ring a bell to > anyone? Is this a known Wildfly issue by any chance? > >>>>>>> > >>>>>>> [1] https://gist.github.com/galderz/bd74cebfc840ef3ae284 > >>>>>>> > >>>>>>> Cheers, > >>>>>>> -- > >>>>>>> Galder Zamarreño > >>>>>>> gal...@redhat.com > >>>>>>> twitter.com/galderz > >>>>>>> > >>>>>>> Project Lead, Escalante > >>>>>>> http://escalante.io > >>>>>>> > >>>>>>> Engineer, Infinispan > >>>>>>> http://infinispan.org > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> jboss-as7-dev mailing list > >>>>>>> jboss-as7-...@lists.jboss.org > >>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Galder Zamarreño > >>>>>> gal...@redhat.com > >>>>>> twitter.com/galderz > >>>>>> > >>>>>> Project Lead, Escalante > >>>>>> http://escalante.io > >>>>>> > >>>>>> Engineer, Infinispan > >>>>>> http://infinispan.org > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Galder Zamarreño > >>>>> gal...@redhat.com > >>>>> twitter.com/galderz > >>>>> > >>>>> Project Lead, Escalante > >>>>> http://escalante.io > >>>>> > >>>>> Engineer, Infinispan > >>>>> http://infinispan.org > >>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Galder Zamarreño > >>>> gal...@redhat.com > >>>> twitter.com/galderz > >>>> > >>>> Project Lead, Escalante > >>>> http://escalante.io > >>>> > >>>> Engineer, Infinispan > >>>> http://infinispan.org > >>>> > >>>> > >>>> _______________________________________________ > >>>> infinispan-dev mailing list > >>>> infinispan-dev@lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> > >>> -- > >>> Galder Zamarreño > >>> gal...@redhat.com > >>> twitter.com/galderz > >>> > >>> Project Lead, Escalante > >>> http://escalante.io > >>> > >>> Engineer, Infinispan > >>> http://infinispan.org > >>> > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev@lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev@lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> > >> -- > >> Galder Zamarreño > >> gal...@redhat.com > >> twitter.com/galderz > >> > >> Project Lead, Escalante > >> http://escalante.io > >> > >> Engineer, Infinispan > >> http://infinispan.org > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev@lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev@lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > -- > > Galder Zamarreño > > gal...@redhat.com > > twitter.com/galderz > > > > Project Lead, Escalante > > http://escalante.io > > > > Engineer, Infinispan > > http://infinispan.org > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Galder Zamarreño > gal...@redhat.com > twitter.com/galderz > > Project Lead, Escalante > http://escalante.io > > Engineer, Infinispan > http://infinispan.org > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev >
_______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
