I'm part of Apache Lucene and I try to make every release meet quality standards, but Solr tests are quite complex and have had a long history of being very difficult to fix. If you search mailing list archives you'll see at least a few attempts and continuous efforts from various people aiming at stabilizing these test.
This said, if you're asking whether there was a "green" light before the release then yes -- anybody who votes for the release to proceed ran the smoketester and this in turn runs all the tests. But machines and environments vary. I don't know anybody who'd run development runs as root, for example (which you apparently did). The problems you initially experienced were your environment setup problems (different compile JDK compared to runtime JDK), they don't have much to do with Solr tests too. Dawid On Tue, May 5, 2015 at 2:48 PM, Per Steffensen <[email protected]> wrote: > I believe a JIRA will be about fixing a problem. But right now it is not so > much about fixing a problem, it is about telling if there is a problem in > the code to fix, or if it is just us doing something wrong. We want to be > able to run the test-suite with the same test-results as you get at Apache > CI. So if this particular test also fail at Apache CI for version 5.1.0 we > have no problem. It is just that I would expect you to have seen a green > test-suite in Apache CI on 5.1.0 code before releasing it. The fact that we > do not see a green test-suite on 5.1.0 make us wonder why - do we run the > test-suite correctly, or ... ? > > Can you confirm that you have seen a green test-suite at Apache CI on 5.1.0 > code? Because then we have a problem in my organization, and there is > nothing to fix on Apache side, and therefore no JIRA to open. > > If the test is red also on you side, and it is not because we run the > test-suite in a wrong way, of course it should be corrected, and that would > call for a JIRA. But right now we are not sure. > > Regards, Per Steffensen > > > On 05/05/15 14:08, Mark Miller wrote: > > +1 - I'd file JIRAs. > > - Mark > > On Tue, May 5, 2015 at 6:13 AM Dawid Weiss <[email protected]> > wrote: >> >> The assertion alone isn't helpful. Solr tests dump tons of logs along >> the way, if you can copy these to a text file and then create a jira >> issue (reference it here) then perhaps somebody can look into the >> cause of the problem (or you can do it -- it's open source after all >> :). >> >> https://issues.apache.org/jira/secure/Dashboard.jspa >> >> Dawid >> >> On Tue, May 5, 2015 at 11:29 AM, Kåre Brandborg <[email protected]> wrote: >> > I’m a colleague of Per from who I’ve taken over the task to try to get >> > our test environment to build Solr 5.1 and compile and run a test suite >> > with >> > green lights. >> > >> > I’ll try to elaborate a little more about our progress. >> > >> > We are currently using Teamcity CI and we are running our tests on an >> > Ubuntu 12.04 x64 with jdk7u76 (x64) and ant 1.9.4. >> > >> > We have made a single change to the: ./lucene/ivy-settings.xml file (to >> > point it to use our internal repository to resolve artifacts. >> > >> > I’ve observed that the following test is failing for us on every run >> > (Have made 5 runs so far on the configuration above): >> > >> > 2> 1490839 T6144 oasc.CachingDirectoryFactory.close Closing directory: >> > /opt/buildagent/work/6906da56abce9b00/solr/build/solr-core/test/J1/temp/solr.core.TestCoreDiscovery >> > 93911594584047D7-001/tempDir-001/core2/core2/index >> > 2> 1490840 T6144 oas.SolrTestCaseJ4.tearDown ###Ending >> > testCoreDirCantRead >> > 2> NOTE: reproduce with: ant test -Dtestcase=TestCoreDiscovery >> > -Dtests.method=testCoreDirCantRead -Dtests.seed=93911594584047D7 >> > -Dtests.slow=true -Dtests.locale=ar_KW -Dtests.timezone=Australia/West >> > -Dtests.asserts=true -Dtests.file.encoding=US-ASCII >> > FAILURE 0.72s J1 | TestCoreDiscovery.testCoreDirCantRead <<< >> >> Throwable #1: java.lang.AssertionError >> >> at >> >> __randomizedtesting.SeedInfo.seed([93911594584047D7:EED5C1C227C789A5]:0) >> >> at >> >> org.apache.solr.core.TestCoreDiscovery.testCoreDirCantRead(TestCoreDiscovery.java:286) >> >> at java.lang.Thread.run(Thread.java:745) >> > 2> 1490889 T6144 oas.SolrTestCaseJ4.setUp ###Starting >> > testAlternateCoreDir >> > >> > And we are running ant with the following parameter: >> > >> > ant -f build.xml -Dtests.haltonfailure=false test >> > >> > Looking for any help on what can be causing these tests to fail. We >> > suspect a misconfiguration in our environment but are unsure where to look. >> > >> > Thanks >> > >> > Kaare Brandborg >> > >> > >> >> On 04 May 2015, at 14:30, Dawid Weiss <[email protected]> >> >> wrote: >> >> >> >>> [junit4] ERROR 3.81s J2 | >> >>> TestDirectoryTaxonomyWriter.testConcurrency >> >>> <<< >> >>> [junit4] > Throwable #1: java.lang.NoSuchMethodError: >> >>> >> >>> java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView; >> >> >> >> Sorry about the delay. This indicates your code was compiled with >> >> JDK1.8 but is executed with Java < 1.8. This method's signature used >> >> to be an interface, but is a covariant pointing at a specialized >> >> subclass in 1.8. >> >> >> >> You need to compile the code with the version of Java you intend to >> >> run with. Things will in general work if you compile with an older >> >> version and try to run with a newer version but not the other way >> >> around. >> >> >> >> You can cross-compile with javac from a newer version of the JDK to an >> >> older version but you'd have to specify bootclasspath to the older >> >> version anyway (bytecode/source flag in javac is not enough) so >> >> there's really no sensible reason to do it in the first place. >> >> >> >> Dawid >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [email protected] >> >> For additional commands, e-mail: [email protected] >> >> >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: [email protected] >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
