Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
: But this could just be a stupid bug in smokeTester. All it looks for : is Started SocketConnector@0.0.0.0:8983 in the server's stderr : output. Maybe this is too brittle? Solr's SLF4J logging recently changed to use log4j as the binding in th example tests instead of JUL. With that change, the log format changed significantly (and there may havebeen some states on trunk when there was no logging configured for example? ... not sure) perhaps that explains how an unexpected edge case was triggered in the smoker checker? -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
On Mon, Apr 1, 2013 at 12:32 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : But this could just be a stupid bug in smokeTester. All it looks for : is Started SocketConnector@0.0.0.0:8983 in the server's stderr : output. Maybe this is too brittle? Solr's SLF4J logging recently changed to use log4j as the binding in th example tests instead of JUL. With that change, the log format changed significantly (and there may havebeen some states on trunk when there was no logging configured for example? ... not sure) perhaps that explains how an unexpected edge case was triggered in the smoker checker? Good idea! So this may have just been a transient thing ... we can wait and see if it happens again. Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
Hmm cascading errors. First, the 4.x smoke tester failed because Solr's example (java -jar start.jar) took more than 30 minutes to start: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/61/console But then because of a bug in the smoke tester, it left this server running, which then caused future smoke testers to fail since they can't bind to the port. So: * I killed the leftover java -jar start.jar on Jenkins * I'll fix smoke tester to not leave leftover processes Still not sure why the Solr example took more than 30 minutes to start though ... could just be a bug in the smoke tester. Mike McCandless http://blog.mikemccandless.com On Sun, Mar 31, 2013 at 4:43 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.2.1/11/ No tests ran. Build Log: [...truncated 32490 lines...] prepare-release-no-sign: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease [copy] Copying 401 files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/lucene [copy] Copying 194 files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/solr [exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6 [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7 [exec] NOTE: output encoding is US-ASCII [exec] [exec] Load release URL file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/... [exec] [exec] Test Lucene... [exec] test basics... [exec] get KEYS [exec] 0.1 MB [exec] check changes HTML... [exec] download lucene-4.2.1-src.tgz... [exec] 26.7 MB [exec] verify md5/sha1 digests [exec] download lucene-4.2.1.tgz... [exec] 48.1 MB [exec] verify md5/sha1 digests [exec] download lucene-4.2.1.zip... [exec] 57.7 MB [exec] verify md5/sha1 digests [exec] unpack lucene-4.2.1.tgz... [exec] verify JAR/WAR metadata... [exec] test demo with 1.6... [exec] got 5450 hits for query lucene [exec] test demo with 1.7... [exec] got 5450 hits for query lucene [exec] check Lucene's javadoc JAR [exec] unpack lucene-4.2.1.zip... [exec] verify JAR/WAR metadata... [exec] test demo with 1.6... [exec] got 5450 hits for query lucene [exec] test demo with 1.7... [exec] got 5450 hits for query lucene [exec] check Lucene's javadoc JAR [exec] unpack lucene-4.2.1-src.tgz... [exec] make sure no JARs/WARs in src dist... [exec] run ant validate [exec] run tests w/ Java 6... [exec] test demo with 1.6... [exec] got 222 hits for query lucene [exec] generate javadocs w/ Java 6... [exec] run tests w/ Java 7... [exec] test demo with 1.7... [exec] got 222 hits for query lucene [exec] generate javadocs w/ Java 7... [exec] [exec] Crawl/parse... [exec] [exec] Verify... [exec] [exec] Test Solr... [exec] test basics... [exec] get KEYS [exec] 0.1 MB [exec] check changes HTML... [exec] download solr-4.2.1-src.tgz... [exec] 30.3 MB [exec] verify md5/sha1 digests [exec] download solr-4.2.1.tgz... [exec] 111.0 MB [exec] verify md5/sha1 digests [exec] download solr-4.2.1.zip... [exec] 115.5 MB [exec] verify md5/sha1 digests [exec] unpack solr-4.2.1.tgz... [exec] verify JAR/WAR metadata... [exec] **WARNING**: skipping check of /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1/contrib/dataimporthandler/lib/activation-1.1.jar: it has javax.* classes [exec] **WARNING**: skipping check of /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1/contrib/dataimporthandler/lib/mail-1.4.1.jar: it has javax.* classes [exec] make sure WAR file has no javax.* or java.* classes... [exec] copying unpacked distribution for Java 6 ... [exec] test solr example w/ Java 6... [exec] start Solr instance (log=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1-java6/solr-example.log)... [exec] Startup failed; see log /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1-java6/solr-example.log [exec] 2013-03-31 20:43:07.748:INFO:oejs.Server:jetty-8.1.8.v20121106 [exec]
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
Good point. I'll take down all the 4.2.1 jobs. Steve On Mar 31, 2013, at 6:15 PM, Robert Muir rcm...@gmail.com wrote: Maybe the fact we now have this 4.2.1-SmokeRelease job (didnt the vote pass?) created a situation where two smoke-testing jobs (e.g. 5.x and 4.2.1 or something) were running concurrently and both tried to bind to the same port? On Sun, Mar 31, 2013 at 5:29 PM, Michael McCandless luc...@mikemccandless.com wrote: Hmm cascading errors. First, the 4.x smoke tester failed because Solr's example (java -jar start.jar) took more than 30 minutes to start: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/61/console But then because of a bug in the smoke tester, it left this server running, which then caused future smoke testers to fail since they can't bind to the port. So: * I killed the leftover java -jar start.jar on Jenkins * I'll fix smoke tester to not leave leftover processes Still not sure why the Solr example took more than 30 minutes to start though ... could just be a bug in the smoke tester. Mike McCandless http://blog.mikemccandless.com On Sun, Mar 31, 2013 at 4:43 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.2.1/11/ No tests ran. Build Log: [...truncated 32490 lines...] prepare-release-no-sign: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease [copy] Copying 401 files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/lucene [copy] Copying 194 files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/solr [exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6 [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7 [exec] NOTE: output encoding is US-ASCII [exec] [exec] Load release URL file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/... [exec] [exec] Test Lucene... [exec] test basics... [exec] get KEYS [exec] 0.1 MB [exec] check changes HTML... [exec] download lucene-4.2.1-src.tgz... [exec] 26.7 MB [exec] verify md5/sha1 digests [exec] download lucene-4.2.1.tgz... [exec] 48.1 MB [exec] verify md5/sha1 digests [exec] download lucene-4.2.1.zip... [exec] 57.7 MB [exec] verify md5/sha1 digests [exec] unpack lucene-4.2.1.tgz... [exec] verify JAR/WAR metadata... [exec] test demo with 1.6... [exec] got 5450 hits for query lucene [exec] test demo with 1.7... [exec] got 5450 hits for query lucene [exec] check Lucene's javadoc JAR [exec] unpack lucene-4.2.1.zip... [exec] verify JAR/WAR metadata... [exec] test demo with 1.6... [exec] got 5450 hits for query lucene [exec] test demo with 1.7... [exec] got 5450 hits for query lucene [exec] check Lucene's javadoc JAR [exec] unpack lucene-4.2.1-src.tgz... [exec] make sure no JARs/WARs in src dist... [exec] run ant validate [exec] run tests w/ Java 6... [exec] test demo with 1.6... [exec] got 222 hits for query lucene [exec] generate javadocs w/ Java 6... [exec] run tests w/ Java 7... [exec] test demo with 1.7... [exec] got 222 hits for query lucene [exec] generate javadocs w/ Java 7... [exec] [exec] Crawl/parse... [exec] [exec] Verify... [exec] [exec] Test Solr... [exec] test basics... [exec] get KEYS [exec] 0.1 MB [exec] check changes HTML... [exec] download solr-4.2.1-src.tgz... [exec] 30.3 MB [exec] verify md5/sha1 digests [exec] download solr-4.2.1.tgz... [exec] 111.0 MB [exec] verify md5/sha1 digests [exec] download solr-4.2.1.zip... [exec] 115.5 MB [exec] verify md5/sha1 digests [exec] unpack solr-4.2.1.tgz... [exec] verify JAR/WAR metadata... [exec] **WARNING**: skipping check of /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1/contrib/dataimporthandler/lib/activation-1.1.jar: it has javax.* classes [exec] **WARNING**: skipping check of /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1/contrib/dataimporthandler/lib/mail-1.4.1.jar: it has javax.* classes [exec] make sure WAR file has no javax.* or java.* classes...
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
I think there's only one executor on the lucene slave, though, so no concurrent jobs. On Mar 31, 2013, at 6:15 PM, Robert Muir rcm...@gmail.com wrote: Maybe the fact we now have this 4.2.1-SmokeRelease job (didnt the vote pass?) created a situation where two smoke-testing jobs (e.g. 5.x and 4.2.1 or something) were running concurrently and both tried to bind to the same port? On Sun, Mar 31, 2013 at 5:29 PM, Michael McCandless luc...@mikemccandless.com wrote: Hmm cascading errors. First, the 4.x smoke tester failed because Solr's example (java -jar start.jar) took more than 30 minutes to start: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/61/console But then because of a bug in the smoke tester, it left this server running, which then caused future smoke testers to fail since they can't bind to the port. So: * I killed the leftover java -jar start.jar on Jenkins * I'll fix smoke tester to not leave leftover processes Still not sure why the Solr example took more than 30 minutes to start though ... could just be a bug in the smoke tester. Mike McCandless http://blog.mikemccandless.com On Sun, Mar 31, 2013 at 4:43 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.2.1/11/ No tests ran. Build Log: [...truncated 32490 lines...] prepare-release-no-sign: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease [copy] Copying 401 files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/lucene [copy] Copying 194 files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/solr [exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6 [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7 [exec] NOTE: output encoding is US-ASCII [exec] [exec] Load release URL file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeRelease/... [exec] [exec] Test Lucene... [exec] test basics... [exec] get KEYS [exec] 0.1 MB [exec] check changes HTML... [exec] download lucene-4.2.1-src.tgz... [exec] 26.7 MB [exec] verify md5/sha1 digests [exec] download lucene-4.2.1.tgz... [exec] 48.1 MB [exec] verify md5/sha1 digests [exec] download lucene-4.2.1.zip... [exec] 57.7 MB [exec] verify md5/sha1 digests [exec] unpack lucene-4.2.1.tgz... [exec] verify JAR/WAR metadata... [exec] test demo with 1.6... [exec] got 5450 hits for query lucene [exec] test demo with 1.7... [exec] got 5450 hits for query lucene [exec] check Lucene's javadoc JAR [exec] unpack lucene-4.2.1.zip... [exec] verify JAR/WAR metadata... [exec] test demo with 1.6... [exec] got 5450 hits for query lucene [exec] test demo with 1.7... [exec] got 5450 hits for query lucene [exec] check Lucene's javadoc JAR [exec] unpack lucene-4.2.1-src.tgz... [exec] make sure no JARs/WARs in src dist... [exec] run ant validate [exec] run tests w/ Java 6... [exec] test demo with 1.6... [exec] got 222 hits for query lucene [exec] generate javadocs w/ Java 6... [exec] run tests w/ Java 7... [exec] test demo with 1.7... [exec] got 222 hits for query lucene [exec] generate javadocs w/ Java 7... [exec] [exec] Crawl/parse... [exec] [exec] Verify... [exec] [exec] Test Solr... [exec] test basics... [exec] get KEYS [exec] 0.1 MB [exec] check changes HTML... [exec] download solr-4.2.1-src.tgz... [exec] 30.3 MB [exec] verify md5/sha1 digests [exec] download solr-4.2.1.tgz... [exec] 111.0 MB [exec] verify md5/sha1 digests [exec] download solr-4.2.1.zip... [exec] 115.5 MB [exec] verify md5/sha1 digests [exec] unpack solr-4.2.1.tgz... [exec] verify JAR/WAR metadata... [exec] **WARNING**: skipping check of /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1/contrib/dataimporthandler/lib/activation-1.1.jar: it has javax.* classes [exec] **WARNING**: skipping check of /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.2.1/lucene/build/fakeReleaseTmp/unpack/solr-4.2.1/contrib/dataimporthandler/lib/mail-1.4.1.jar: it has javax.* classes [exec] make sure WAR file has no javax.*
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
On Sun, Mar 31, 2013 at 6:23 PM, Robert Muir rcm...@gmail.com wrote: hmm there goes that theory. Maybe its just a leftover process that didn't get killed from a previous smoketester: I think to be safe the python code should always terminate the server it starts with 'kill -9' and nothing else! It does kill the server ... but there was a bug in that logic such that if the 30 minute startup wait elapsed it failed to kill it. Which is curious ... it seems to mean that the server took 30 minutes, didn't seem to start, but did in fact start (after 30 minutes) and bind to the port. But this could just be a stupid bug in smokeTester. All it looks for is Started SocketConnector@0.0.0.0:8983 in the server's stderr output. Maybe this is too brittle? Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.2.1 - Build # 11 - Still Failing
On Sun, Mar 31, 2013 at 6:42 PM, Michael McCandless luc...@mikemccandless.com wrote: On Sun, Mar 31, 2013 at 6:23 PM, Robert Muir rcm...@gmail.com wrote: hmm there goes that theory. Maybe its just a leftover process that didn't get killed from a previous smoketester: I think to be safe the python code should always terminate the server it starts with 'kill -9' and nothing else! It does kill the server ... but there was a bug in that logic such that if the 30 minute startup wait elapsed it failed to kill it. Which is curious ... it seems to mean that the server took 30 minutes, didn't seem to start, but did in fact start (after 30 minutes) and bind to the port. maybe its related to the blackhole :)
