Re: new JDBC server test cases seems failed ?
Noticed that Nan’s PR is not related to SQL, but the JDBC test suites got executed. Then I checked PRs of all those Jenkins builds that failed because of the JDBC suites, it turns out that none of them touched SQL code. The JDBC code is only contained in the assembly file when the hive-thriftserver build profile is enabled. So it seems that the root cause is related to Maven build changes that makes the JDBC suites always get executed and fail because JDBC code isn't included in the assembly jar. This also explains why I can’t reproduce it locally (I always enable hive-thriftserver profile), and why once the build fail, all JDBC suites fail together. Working on a patch to fix this. Thanks to Patrick for helping debugging this! On Jul 28, 2014, at 10:07 AM, Cheng Lian l...@databricks.com wrote: I’m looking into this, will fix this ASAP, sorry for the inconvenience. On Jul 28, 2014, at 9:47 AM, Patrick Wendell pwend...@gmail.com wrote: I'm going to revert it again - Cheng can you try to look into this? Thanks. On Sun, Jul 27, 2014 at 6:06 PM, Nan Zhu zhunanmcg...@gmail.com wrote: it's 20 minutes ago https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17259/consoleFull -- Nan Zhu On Sunday, July 27, 2014 at 8:53 PM, Michael Armbrust wrote: How recent is this? We've already reverted this patch once due to failing tests. It would be helpful to include a link to the failed build. If its failing again we'll have to revert again. On Sun, Jul 27, 2014 at 5:26 PM, Nan Zhu zhunanmcg...@gmail.com (mailto:zhunanmcg...@gmail.com) wrote: Hi, all It seems that the JDBC test cases are failed unexpectedly in Jenkins? [info] - test query execution against a Hive Thrift server *** FAILED *** [info] java.sql.SQLException: Could not open connection to jdbc:hive2://localhost:45518/: java.net.ConnectException: Connection refused [info] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:146) [info] at org.apache.hive.jdbc.HiveConnection.init(HiveConnection.java:123) [info] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply$mcV$sp(HiveThriftServer2Suite.scala:110) [info] at org.apache.spark.sql.hive.thri ftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) [info] ... [info] Cause: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused [info] at org.apache.thrift.transport.TSocket.open(TSocket.java:185) [info] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) [info] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [info] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:144) [info] at org.apache.hive.jdbc.HiveConnection.init(HiveConnection.java:123) [info] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at org.apache.spark.sql.hive.thriftserver.H iveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) [info] ... [info] Cause: java.net.ConnectException: Connection refused [info] at java.net.PlainSocketImpl.socketConnect(Native Method) [info] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [info] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [info] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [info] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [info] at java.net.Socket.connect(Socket.java:579) [info] at org.apache.thrift.transport.TSocket.open(TSocket.java:180) [info] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) [info] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [info] at org.apache.hive.jdbc.HiveConn ection.openTransport(HiveConnection.java:144) [info] ... [info] CliSuite: Executing: create table hive_test1(key int, val string);, expecting output: OK [warn] four warnings found [warn] Note:
Re: new JDBC server test cases seems failed ?
How recent is this? We've already reverted this patch once due to failing tests. It would be helpful to include a link to the failed build. If its failing again we'll have to revert again. On Sun, Jul 27, 2014 at 5:26 PM, Nan Zhu zhunanmcg...@gmail.com wrote: Hi, all It seems that the JDBC test cases are failed unexpectedly in Jenkins? [info] - test query execution against a Hive Thrift server *** FAILED *** [info] java.sql.SQLException: Could not open connection to jdbc:hive2://localhost:45518/: java.net.ConnectException: Connection refused [info] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:146) [info] at org.apache.hive.jdbc.HiveConnection.init(HiveConnection.java:123) [info] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply$mcV$sp(HiveThriftServer2Suite.scala:110) [info] at org.apache.spark.sql.hive.thri ftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) [info] ... [info] Cause: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused [info] at org.apache.thrift.transport.TSocket.open(TSocket.java:185) [info] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) [info] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [info] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:144) [info] at org.apache.hive.jdbc.HiveConnection.init(HiveConnection.java:123) [info] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at org.apache.spark.sql.hive.thriftserver.H iveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) [info] ... [info] Cause: java.net.ConnectException: Connection refused [info] at java.net.PlainSocketImpl.socketConnect(Native Method) [info] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [info] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [info] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [info] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [info] at java.net.Socket.connect(Socket.java:579) [info] at org.apache.thrift.transport.TSocket.open(TSocket.java:180) [info] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) [info] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [info] at org.apache.hive.jdbc.HiveConn ection.openTransport(HiveConnection.java:144) [info] ... [info] CliSuite: Executing: create table hive_test1(key int, val string);, expecting output: OK [warn] four warnings found [warn] Note: /home/jenkins/workspace/SparkPullRequestBuilder@4/core/src/test/java/org/apache/spark/JavaAPISuite.java uses or overrides a deprecated API. [warn] Note: Recompile with -Xlint:deprecation for details. [info] - simple commands *** FAILED *** [info] java.lang.AssertionError: assertion failed: Didn't find OK in the output: [info] at scala.Predef$.assert(Predef.scala:179) [info] at org.apache.spark.sql.hive.thriftserver.TestUtils$class.waitForQuery(TestUtils.scala:70) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite.waitForQuery(CliSuite.scala:25) [info] at org.apache.spark.sql.hive.thriftserver.TestUtils$class.executeQuery(TestUtils.scala:62) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite.executeQuery(CliSuite.scala:25) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite $$anonfun$1.apply$mcV$sp(CliSuite.scala:53) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite$$anonfun$1.apply(CliSuite.scala:51) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite$$anonfun$1.apply(CliSuite.scala:51) [info] at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) [info] at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) [log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See
Re: new JDBC server test cases seems failed ?
I'm going to revert it again - Cheng can you try to look into this? Thanks. On Sun, Jul 27, 2014 at 6:06 PM, Nan Zhu zhunanmcg...@gmail.com wrote: it's 20 minutes ago https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17259/consoleFull -- Nan Zhu On Sunday, July 27, 2014 at 8:53 PM, Michael Armbrust wrote: How recent is this? We've already reverted this patch once due to failing tests. It would be helpful to include a link to the failed build. If its failing again we'll have to revert again. On Sun, Jul 27, 2014 at 5:26 PM, Nan Zhu zhunanmcg...@gmail.com (mailto:zhunanmcg...@gmail.com) wrote: Hi, all It seems that the JDBC test cases are failed unexpectedly in Jenkins? [info] - test query execution against a Hive Thrift server *** FAILED *** [info] java.sql.SQLException: Could not open connection to jdbc:hive2://localhost:45518/: java.net.ConnectException: Connection refused [info] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:146) [info] at org.apache.hive.jdbc.HiveConnection.init(HiveConnection.java:123) [info] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply$mcV$sp(HiveThriftServer2Suite.scala:110) [info] at org.apache.spark.sql.hive.thri ftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite$$anonfun$1.apply(HiveThriftServer2Suite.scala:107) [info] ... [info] Cause: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused [info] at org.apache.thrift.transport.TSocket.open(TSocket.java:185) [info] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) [info] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [info] at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:144) [info] at org.apache.hive.jdbc.HiveConnection.init(HiveConnection.java:123) [info] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) [info] at java.sql.DriverManager.getConnection(DriverManager.java:571) [info] at java.sql.DriverManager.getConnection(DriverManager.java:215) [info] at org.apache.spark.sql.hive.thriftserver.H iveThriftServer2Suite.getConnection(HiveThriftServer2Suite.scala:131) [info] at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2Suite.createStatement(HiveThriftServer2Suite.scala:134) [info] ... [info] Cause: java.net.ConnectException: Connection refused [info] at java.net.PlainSocketImpl.socketConnect(Native Method) [info] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [info] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [info] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [info] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [info] at java.net.Socket.connect(Socket.java:579) [info] at org.apache.thrift.transport.TSocket.open(TSocket.java:180) [info] at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248) [info] at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [info] at org.apache.hive.jdbc.HiveConn ection.openTransport(HiveConnection.java:144) [info] ... [info] CliSuite: Executing: create table hive_test1(key int, val string);, expecting output: OK [warn] four warnings found [warn] Note: /home/jenkins/workspace/SparkPullRequestBuilder@4/core/src/test/java/org/apache/spark/JavaAPISuite.java uses or overrides a deprecated API. [warn] Note: Recompile with -Xlint:deprecation for details. [info] - simple commands *** FAILED *** [info] java.lang.AssertionError: assertion failed: Didn't find OK in the output: [info] at scala.Predef$.assert(Predef.scala:179) [info] at org.apache.spark.sql.hive.thriftserver.TestUtils$class.waitForQuery(TestUtils.scala:70) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite.waitForQuery(CliSuite.scala:25) [info] at org.apache.spark.sql.hive.thriftserver.TestUtils$class.executeQuery(TestUtils.scala:62) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite.executeQuery(CliSuite.scala:25) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite $$anonfun$1.apply$mcV$sp(CliSuite.scala:53) [info] at org.apache.spark.sql.hive.thriftserver.CliSuite$$anonfun$1.apply(CliSuite.scala:51) [info]
