There seems to be something wrong with this code: public static List<FrameworkMethod> computeTestMethodsForClass(final Class klass, final List<FrameworkMethod> defaultMethods) { List<FrameworkMethod> list = defaultMethods; String methodName = System.getProperty("test.method"); if (methodName == null) { LOG.info("No test.method specified. using default methods."); } else { LOG.info("Picked up test.method={}", methodName); try { list = Arrays.asList(new FrameworkMethod(klass.getMethod(methodName))); } catch (NoSuchMethodException nsme) { LOG.warn("{} does not have test.method={}. failing to default methods.", klass.getName(), methodName); } } return list; }
This is in JUnit4ZKTestRunner. Perhaps a race? -Flavio > On 29 Jul 2016, at 05:12, Michael Han <h...@cloudera.com> wrote: > > While triage Jenkins builds, I find out that some tests could be in the > state of 'Skipped'. This build is an example, in particular this is the > skipped test > https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/testReport/org.apache.zookeeper.test/QuorumTest/testSessionMove/ > > It looks like if there is any test that's in 'skipped' state, then the > build will be marked as failure, regardless if all other tests passed or > not. > Do we know what could cause a test being skipped? The log said 'No > test.method specified.' but this does not deterministically happen for any > specific tests. > > > Here is the log of the skipped test: > Skipped > > org.apache.zookeeper.test.QuorumTest.testSessionMove > Skipped for the past 82 builds (Since > <https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/>#3221 ) > Took 1 ms. > <https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/testReport/org.apache.zookeeper.test/QuorumTest/testSessionMove/history> > Standard Output > > 2016-07-29 01:06:54,754 [myid:] - INFO [main:JUnit4ZKTestRunner@47] - > No test.method specified. using default methods. > 2016-07-29 01:06:54,813 [myid:] - INFO [main:JUnit4ZKTestRunner@47] - > No test.method specified. using default methods. > 2016-07-29 01:06:54,827 [myid:] - INFO [main:PortAssignment@151] - > Test process 4/8 using ports from 19300 - 21992. > 2016-07-29 01:06:54,829 [myid:] - INFO [main:PortAssignment@85] - > Assigned port 19301 from range 19300 - 21992. > 2016-07-29 01:06:54,835 [myid: > ...[truncated 2825034 chars]... > il@243] - 127.0.0.1:19566 is no longer accepting client connections > 2016-07-29 01:09:44,398 [myid:] - INFO [main:FourLetterWordMain@85] - > connecting to 127.0.0.1 19569 > 2016-07-29 01:09:44,398 [myid:] - INFO [main:QuorumUtil@243] - > 127.0.0.1:19569 is no longer accepting client connections > 2016-07-29 01:09:44,399 [myid:] - INFO [main:ZKTestCase$1@65] - > SUCCEEDED testFollowersStartAfterLeader > 2016-07-29 01:09:44,399 [myid:] - INFO [main:ZKTestCase$1@60] - > FINISHED testFollowersStartAfterLeader > > > > > On Thu, Jul 28, 2016 at 6:20 PM, Apache Jenkins Server < > jenk...@builds.apache.org> wrote: > >> Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2492 >> Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/ >> >> >> ################################################################################### >> ########################## LAST 60 LINES OF THE CONSOLE >> ########################### >> [...truncated 443101 lines...] >> [exec] -1 tests included. The patch doesn't appear to include >> any new or modified tests. >> [exec] Please justify why no new tests are >> needed for this patch. >> [exec] Also please list what manual steps >> were performed to verify this patch. >> [exec] >> [exec] +1 javadoc. The javadoc tool did not generate any warning >> messages. >> [exec] >> [exec] +1 javac. The applied patch does not increase the total >> number of javac compiler warnings. >> [exec] >> [exec] +1 findbugs. The patch does not introduce any new >> Findbugs (version 2.0.3) warnings. >> [exec] >> [exec] +1 release audit. The applied patch does not increase the >> total number of release audit warnings. >> [exec] >> [exec] +1 core tests. The patch passed core unit tests. >> [exec] >> [exec] +1 contrib tests. The patch passed contrib unit tests. >> [exec] >> [exec] Test results: >> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//testReport/ >> [exec] Findbugs warnings: >> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html >> [exec] Console output: >> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//console >> [exec] >> [exec] This message is automatically generated. >> [exec] >> [exec] >> [exec] >> ====================================================================== >> [exec] >> ====================================================================== >> [exec] Adding comment to Jira. >> [exec] >> ====================================================================== >> [exec] >> ====================================================================== >> [exec] >> [exec] >> [exec] Comment added. >> [exec] 69951929bc7887930c177e2a8617f3c2a0800e52 logged out >> [exec] >> [exec] >> [exec] >> ====================================================================== >> [exec] >> ====================================================================== >> [exec] Finished build. >> [exec] >> ====================================================================== >> [exec] >> ====================================================================== >> [exec] >> [exec] >> >> BUILD FAILED >> /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk/build.xml:1605: >> exec returned: 1 >> >> Total time: 21 minutes 17 seconds >> Build step 'Execute shell' marked build as failure >> Archiving artifacts >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> Recording test results >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> [description-setter] Description set: ZOOKEEPER-2492 >> Email was triggered for: Failure - Any >> Sending email for trigger: Failure - Any >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> Setting >> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 >> >> >> >> >> ################################################################################### >> ############################## FAILED TESTS (if any) >> ############################## >> All tests passed > > > > > -- > Cheers > Michael.