Okay, I finally got some time to debug this. This is what seems to be the problem:
1. In our master branch, the maven-surefire plugin version is 2.22.2 <https://github.com/apache/phoenix/blob/70baf4c6805eb46420dd1fbaf71a093ab7d84a8b/pom.xml#L126> and in our 4.x branches, it is 2.20 <https://github.com/apache/phoenix/blob/69448a845e843628558e17032287fe9e2bb6c5cf/pom.xml#L127>. Looks like PHOENIX-5459 <https://issues.apache.org/jira/browse/PHOENIX-5459> was not applied to 4.x branches. Was this just a miss, or did we not apply to 4.x branches on purpose? 2. In some of our build hosts, we use the OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3), and in others we use Java(TM) SE Runtime Environment (build 1.8.0_191-b12). See JDK-11-build <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull> vs JDK-8-build <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull> . 3. The "java.lang.NoClassDefFoundError: java/sql/SQLException: java.sql.SQLException" was a known issue (SUREFIRE-1265 <https://issues.apache.org/jira/browse/SUREFIRE-1265>) for Java runtime 1.9+ and was fixed in version 2.20.1. 4. The builds that fail with the above error are only for 4.x branches + JDK 11. Here are recent builds satisfying this combination (old surefire and new Java version) and failing with this error: 3095 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095/consoleFull>, 3094 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3094/consoleFull>, 3093 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3093/consoleFull>, 3091 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>. 5. Builds on the master branch (surefire plugin 2.22.2) + JDK 11, do not fail with this error, for ex: 3089 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3089/consoleFull>, 3088 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3088/consoleFull>, 3092 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3092/consoleFull> . Also, builds on 4.x branches (surefire plugin 2.20) + JDK 8 also do not fail with this error, for ex: 3100 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>, 3099 <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3099/consoleFull> It looks like the build issue can be fixed if we apply PHOENIX-5459 to 4.x branches since that would bump up the surefire plugin version + support running tests on JDK11. If PHOENIX-5459 is not be included in 4.x branches for some reason, we should at least bump up the surefire plugin version on 4.x branches. Thoughts? On Thu, Nov 7, 2019 at 9:35 AM Josh Elser <[email protected]> wrote: > What have you been able to find so far, Chinmay? > > For example, > > * Does every PreCommit build fail? If not, are we limited to a certain > host(s) that causes failure? > * Do local builds still work? > * What's different (maven+java versions) from the build hosts to local > builds? > > With coordination from Infra, I think it is possible (or was possible in > the past) to get SSH access to the build workers, but it's rare that > this is actually required. > > On 11/6/19 1:21 PM, Chinmay Kulkarni wrote: > > The Precommit build has been broken for a while now. Recent builds are > > failing with > > > > [ERROR] Failed to execute goal > > org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) > on > > project phoenix-core: Execution default-test of goal > > org.apache.maven.plugins:maven-surefire-plugin:2.20:test failed: > > java.lang.NoClassDefFoundError: java/sql/SQLException: > java.sql.SQLException > > > > I am unaware of how to fix this or even get access to the build > > configurations/build machine itself. We are committing patches without > > getting green builds and that is not a good idea (but we don't have an > > option until the build is fixed either). > > > > Does anyone know how to debug this? > > > -- Chinmay Kulkarni
