Noticed that Nan’s PR is not related to SQL, but the JDBC test suites got executed. Then I checked PRs of all those Jenkins builds that failed because of the JDBC suites, it turns out that none of them touched SQL code. The JDBC code is only contained in the assembly file when the hive-thriftserver build profile is enabled. So it seems that the root cause is related to Maven build changes that makes the JDBC suites always get executed and fail because JDBC code isn't included in the assembly jar. This also explains why I can’t reproduce it locally (I always enable hive-thriftserver profile), and why once the build fail, all JDBC suites fail together.
Working on a patch to fix this. Thanks to Patrick for helping debugging this! On Jul 28, 2014, at 10:07 AM, Cheng Lian <l...@databricks.com> wrote: > I’m looking into this, will fix this ASAP, sorry for the inconvenience. > > On Jul 28, 2014, at 9:47 AM, Patrick Wendell <pwend...@gmail.com> wrote: > >> I'm going to revert it again - Cheng can you try to look into this? Thanks. >> >> On Sun, Jul 27, 2014 at 6:06 PM, Nan Zhu <zhunanmcg...@gmail.com> wrote: >>> it's 20 minutes ago >>> >>> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17259/consoleFull >>> >>> -- >>> Nan Zhu >>> >>> >>> On Sunday, July 27, 2014 at 8:53 PM, Michael Armbrust wrote: >>> >>>> How recent is this? We've already reverted this patch once due to failing >>>> tests. It would be helpful to include a link to the failed build. If its >>>> failing again we'll have to revert again. >>>> >>>> >>>> On Sun, Jul 27, 2014 at 5:26 PM, Nan Zhu <zhunanmcg...@gmail.com >>>> (mailto:zhunanmcg...@gmail.com)> wrote: >>>> >>>>> Hi, all >>>>> >>>>> It seems that the JDBC test cases are failed unexpectedly in Jenkins? >>>>> >>>>> >>>>> [info] - test query execution against a Hive Thrift server *** FAILED *** >>>>> [info] java.sql.SQLException: Could not open connection to >>>>> jdbc:hive2://localhost:45518/: java.net.ConnectException: Connection >>>>> refused [info] at >>>>> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:146) >>>>> [info] at >>>>> org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:123) [info] >>>>> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at >>>>> java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at >>>>> java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at >>>>> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply$mcV$sp(HiveThriftServer2Suite.scala:110) >>>>> [info] at org.apache.spark.sql.hive.thri >>>>> ftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) >>>>> [info] ... [info] Cause: org.apache.thrift.transport.TTransportException: >>>>> java.net.ConnectException: Connection refused [info] at >>>>> org.apache.thrift.transport.TSocket.open(TSocket.java:185) [info] at >>>>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) >>>>> [info] at >>>>> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) >>>>> [info] at >>>>> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:144) >>>>> [info] at >>>>> org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:123) [info] >>>>> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at >>>>> java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at >>>>> java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at >>>>> org.apache.spark.sql.hive.thriftserver.H >>>>> iveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) >>>>> [info] ... [info] Cause: java.net.ConnectException: Connection refused >>>>> [info] at java.net.PlainSocketImpl.socketConnect(Native Method) [info] at >>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) >>>>> [info] at >>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) >>>>> [info] at >>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) >>>>> [info] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>> [info] >>>>> at java.net.Socket.connect(Socket.java:579) [info] at >>>>> org.apache.thrift.transport.TSocket.open(TSocket.java:180) [info] at >>>>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) >>>>> [info] at >>>>> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) >>>>> [info] at org.apache.hive.jdbc.HiveConn >>>>> ection.openTransport(HiveConnection.java:144) [info] ... [info] CliSuite: >>>>> Executing: create table hive_test1(key int, val string);, expecting >>>>> output: >>>>> OK [warn] four warnings found [warn] Note: >>>>> /home/jenkins/workspace/SparkPullRequestBuilder@4/core/src/test/java/org/apache/spark/JavaAPISuite.java >>>>> uses or overrides a deprecated API. [warn] Note: Recompile with >>>>> -Xlint:deprecation for details. [info] - simple commands *** FAILED *** >>>>> [info] java.lang.AssertionError: assertion failed: Didn't find "OK" in the >>>>> output: [info] at scala.Predef$.assert(Predef.scala:179) [info] at >>>>> org.apache.spark.sql.hive.thriftserver.TestUtils$class.waitForQuery(TestUtils.scala:70) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.CliSuite.waitForQuery(CliSuite.scala:25) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.TestUtils$class.executeQuery(TestUtils.scala:62) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.CliSuite.executeQuery(CliSuite.scala:25) >>>>> [info] at org.apache.spark.sql.hive.thriftserver.CliSuite >>>>> $$anonfun$1.apply$mcV$sp(CliSuite.scala:53) [info] at >>>>> org.apache.spark.sql.hive.thriftserver.CliSuite$$anonfun$1.apply(CliSuite.scala:51) >>>>> [info] at >>>>> org.apache.spark.sql.hive.thriftserver.CliSuite$$anonfun$1.apply(CliSuite.scala:51) >>>>> [info] at >>>>> org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) >>>>> [info] at >>>>> org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) >>>>> [log4j:WARN No appenders could be found for logger >>>>> (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j >>>>> system properly. log4j:WARN See >>>>> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. >>>>> 14/07/27 17:06:43 INFO ClientBase: Using Spark's default log4j profile: >>>>> org/apache/spark/log4j-defaults.properties info] ... [info] ScalaTest >>>>> [info] Run completed in 41 seconds, 789 milliseconds. [info] Total number >>>>> of tests run: 2 [info] Suites: completed 2, aborted 0 [info] Tests: >>>>> succeeded 0, failed 2, canceled 0, ignored 0, pending 0 [info] *** 2 TESTS >>>>> FAILED *** >>>>> >>>>> Best, >>>>> >>>>> -- >>>>> Nan Zhu >>>>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig) >>>>> >>>> >>>> >>>> >>> >>> >
