On 04 Feb 2014, at 14:47, Dan Berindei <dan.berin...@gmail.com> wrote:
> > > > On Tue, Feb 4, 2014 at 3:03 PM, Galder Zamarreño <gal...@redhat.com> wrote: > > On 04 Feb 2014, at 13:50, Dan Berindei <dan.berin...@gmail.com> wrote: > > > > > On Tue, Feb 4, 2014 at 2:36 PM, Galder Zamarreño <gal...@redhat.com> wrote: > > Narrowing down the list now, since this is a problem of how our CI is doing > > builds. > > > > These logs are retrieved from [1]. > > > > Dunno how our CI is configured but this is odd. Seems like the build is > > halt due to test failures, but it continues somehow? I mean, the jars are > > not being produced properly, but the build is not halting. > > > > We run the build with -fn (fail-never), so the build should never be halted > > because of a test failure. The configuration is here: > > http://ci.infinispan.org/admin/editRunType.html?id=buildType:bt8&runnerId=RUNNER_1 > > ^ That’s not working as expected, see the build log, my snippets…etc. > > Sorry, I didn't understand what's happening in those snippets. The log shows it quite clearly that after those tests fail, nothing else runs in that module, including producing the jar. It halts. That’s > All I saw was an Ant script that doesn't do what it's supposed to do :) The ant script not doing it’s job is because there modules are not completing the build. There’s a direct correlation between the three modules that fail with tests and the 3 modules that are copying an empty folder instead of the jar. > > I did see some differences in the configuration between the JDK6 and the JDK7 > builds: > * JDK7 uses -fn and JDK6 uses -Dmaven.test.failure.ignore > * JDK7 uses -nsu (no snapshot updates), JDK6 doesn't > > I've changed both builds to use -Dmaven.test.failure.ignore and -nsu, let's > see how it goes. You are solving the wrong problem. > > > > > > > > > > > It’s about time we did the following: > > 1) Any test failures should halt the build there and then. IOW, do not > > continue the build at all. > > > > Will having 100 tests in one run and 2000 tests in another really help? > > As you disable randomly failing tests, and do not integrate commits making > the testsuite fail, these number should even out. > > Not integrating commits that fail every time is easy, not integrating commits > that fail randomly (maybe only in some environments) is trickier. I know it’s tricky, but the only thing we can do is disable those really. I don’t see how keeping them enabled is helping at all. > > > > > > > > 2) Any tests that fail randomly should be disabled. > > > > Let's go ahead and disable all the server tests then? ;) > > Those server tests that are randomly failing should be disabled and looked > at. Those tests that are failing as a result of container not starting are > side effects of things not working properly, and these should not be disabled. > > Why treat the tests that are failing because of a build problem differently? > What about the tests that fail only on IBM JDK6? Disable and indicate that the test fails on IBM JDK6. Once the issue is fixed, reenable it. > > > > > > > > > > Cheers, > > > > [1] https://dl.dropboxusercontent.com/u/6148072/does-not-work.log > > > > On 04 Feb 2014, at 13:30, Galder Zamarreño <gal...@redhat.com> wrote: > > > > > > > > On 04 Feb 2014, at 10:38, Stuart Douglas <stuart.w.doug...@gmail.com> > > > wrote: > > > > > >> It is almost certainly something to do with this: > > >> > > >> <module-def name="org.infinispan.server.rest" > > >> src="${infinispan.server.modules.dir}"> > > >> > > >> <maven-resource-with-classifier group="org.infinispan" > > >> artifact="infinispan-server-rest" classifier="classes" /> > > >> > > >> </module-def> > > >> > > >> I guess sometimes the classes artefact is being attached as a reference > > >> to the classes directory, rather than a reference to a jar, which causes > > >> the issue. > > > > > > Here’s a gist with a subset of the build log [1]. When it works fine, > > > it’s copying a jar, when it’s not, it’s copying an empty folder. > > > > > > However, this is not only happening for the org.infinispan.server.rest > > > module, others show the same issue [2]. What seems to be a pattern is > > > that it only happens with modules that are built by us, it’s not > > > happening for modules coming with the base AS/WF instance. > > > > > > I’ve traced back and this might be due to build failures that are not > > > producing the right jars [3]. > > > > > > @Stuart, this is really our problem. Sorry for the inconvenience! > > > > > > [1] https://gist.github.com/galderz/b9286f385aad1316df51 > > > [2] https://gist.github.com/galderz/9e6a9bd9b18b805db323 > > > [3] https://gist.github.com/galderz/6ab662a1027cd96cbd8c > > > > > >> > > >> Stuart > > >> > > >> > > >> On Tue, Feb 4, 2014 at 11:14 AM, Galder Zamarreño <gal...@redhat.com> > > >> wrote: > > >> > > >> On 04 Feb 2014, at 10:01, Stuart Douglas <stuart.w.doug...@gmail.com> > > >> wrote: > > >> > > >>> > > >>> > > >>> > > >>> On Tue, Feb 4, 2014 at 11:00 AM, Stuart Douglas > > >>> <stuart.w.doug...@gmail.com> wrote: > > >>> Yes, there is nothing in the server code that modified the modules > > >>> directory. > > >>> > > >>> Well, except for the new patching stuff, but that is not really > > >>> relevant here. > > >> > > >> The testsuite AS/WF builds are built out of the distribution build, > > >> which shows the same problem. The distribution we build uses the scripts > > >> we got from AS [1]. > > >> > > >> Do you see anything in there that could be causing this? We are using > > >> maven-antrun-plugin version 1.3, and take into account the lib.xml in > > >> [2]. > > >> > > >> Finally, do you have any suggestions on changes we could make to these > > >> files to further debug the issue? > > >> > > >> Thanks a lot for your help! > > >> > > >> [1] > > >> https://github.com/infinispan/infinispan/blob/master/server/integration/build/build.xml > > >> [2] > > >> https://github.com/infinispan/infinispan/blob/master/server/integration/build/lib.xml > > >> > > >>> > > >>> Stuart > > >>> > > >>> > > >>> Stuart > > >>> > > >>> > > >>> On Tue, Feb 4, 2014 at 10:56 AM, Galder Zamarreño <gal...@redhat.com> > > >>> wrote: > > >>> > > >>> On 04 Feb 2014, at 09:37, Stuart Douglas <stuart.w.doug...@gmail.com> > > >>> wrote: > > >>> > > >>>> This looks like an issue with your environment. The modules directory > > >>>> is static. Wildfly does not contain any code that messes with it. I > > >>>> would say the culprit is probably something in either your build > > >>>> process or your test suite. > > >>> > > >>> Correction, this is happening with AS 7.2.0.Final (Wildfly 8 used > > >>> somewhere else). I guess your answer still applies? > > >>> > > >>> Cheers, > > >>> > > >>>> > > >>>> Stuart > > >>>> > > >>>> > > >>>> On Tue, Feb 4, 2014 at 10:21 AM, Galder Zamarreño <gal...@redhat.com> > > >>>> wrote: > > >>>> Hi all, > > >>>> > > >>>> We’re having issues with our Infinispan Server integration tests, > > >>>> which run within Wildfly 8.0.0.Beta1 (as I’m typing I’m wondering if > > >>>> we should just upgrade it to see if this goes away...?). > > >>>> > > >>>> Quite often some of the runs fail with error message [1]. > > >>>> > > >>>> Having looked at the build environment when a run fails, you see this: > > >>>> > > >>>> -- > > >>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main > > >>>> drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<— a directory??) > > >>>> -rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index > > >>>> -rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml > > >>>> > > >>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main/classes > > >>>> drwxrwxr-x 2 g staff 68B Feb 3 18:41 . > > >>>> drwxrwxr-x 5 g staff 170B Feb 3 18:41 .. > > >>>> > > >>>> $ more > > >>>> modules/system/layers/base/org/infinispan/server/rest/main/module.xml > > >>>> <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”> > > >>>> ... > > >>>> <resource-root path="classes"/> > > >>>> ... > > >>>> > > >>>> This is completely different to what happens with a successful run: > > >>>> > > >>>> -- > > >>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main > > >>>> -rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar (<— > > >>>> a jar file!) > > >>>> -rw-r--r-- 1 g staff 278B Feb 3 19:40 infinispan-classes.jar.index > > >>>> -rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml > > >>>> > > >>>> $ jar tf > > >>>> modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar > > >>>> | grep ExtendedHeaders > > >>>> org/infinispan/rest/configuration/ExtendedHeaders.class > > >>>> > > >>>> $ more > > >>>> modules/system/layers/base/org/infinispan/server/rest/main/module.xml > > >>>> <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”> > > >>>> ... > > >>>> <resource-root path="infinispan-classes.jar”/> > > >>>> — > > >>>> > > >>>> Anyone can explain what is going on here? Does it ring a bell to > > >>>> anyone? Is this a known Wildfly issue by any chance? > > >>>> > > >>>> [1] https://gist.github.com/galderz/bd74cebfc840ef3ae284 > > >>>> > > >>>> Cheers, > > >>>> -- > > >>>> Galder Zamarreño > > >>>> gal...@redhat.com > > >>>> twitter.com/galderz > > >>>> > > >>>> Project Lead, Escalante > > >>>> http://escalante.io > > >>>> > > >>>> Engineer, Infinispan > > >>>> http://infinispan.org > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> jboss-as7-dev mailing list > > >>>> jboss-as7-...@lists.jboss.org > > >>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev > > >>>> > > >>> > > >>> > > >>> -- > > >>> Galder Zamarreño > > >>> gal...@redhat.com > > >>> twitter.com/galderz > > >>> > > >>> Project Lead, Escalante > > >>> http://escalante.io > > >>> > > >>> Engineer, Infinispan > > >>> http://infinispan.org > > >>> > > >>> > > >>> > > >> > > >> > > >> -- > > >> Galder Zamarreño > > >> gal...@redhat.com > > >> twitter.com/galderz > > >> > > >> Project Lead, Escalante > > >> http://escalante.io > > >> > > >> Engineer, Infinispan > > >> http://infinispan.org > > >> > > >> > > > > > > > > > -- > > > Galder Zamarreño > > > gal...@redhat.com > > > twitter.com/galderz > > > > > > Project Lead, Escalante > > > http://escalante.io > > > > > > Engineer, Infinispan > > > http://infinispan.org > > > > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev@lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > -- > > Galder Zamarreño > > gal...@redhat.com > > twitter.com/galderz > > > > Project Lead, Escalante > > http://escalante.io > > > > Engineer, Infinispan > > http://infinispan.org > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Galder Zamarreño > gal...@redhat.com > twitter.com/galderz > > Project Lead, Escalante > http://escalante.io > > Engineer, Infinispan > http://infinispan.org > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño gal...@redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
