lichenglin created ZEPPELIN-1909: ------------------------------------ Summary: spark session with hive support can't be created correctly Key: ZEPPELIN-1909 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1909 Project: Zeppelin Issue Type: Bug Components: Interpreters Affects Versions: 0.7.0 Reporter: lichenglin Priority: Blocker Fix For: 0.7.0
{code} if (useHiveContext()) { if (hiveClassesArePresent()) { Utils.invokeMethod(builder, "enableHiveSupport"); sparkSession = Utils.invokeMethod(builder, "getOrCreate"); logger.info("Created Spark session with Hive support"); } else { Utils.invokeMethod(builder, "config", new Class[]{ String.class, String.class}, new Object[]{ "spark.sql.catalogImplementation", "in-memory"}); sparkSession = Utils.invokeMethod(builder, "getOrCreate"); logger.info("Created Spark session with Hive support"); } } {code} {code} /** * See org.apache.spark.sql.SparkSession.hiveClassesArePresent * @return */ private boolean hiveClassesArePresent() { try { this.getClass().forName("org.apache.spark.sql.hive.HiveSessionState"); this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState"); this.getClass().forName("org.apache.hadoop.hive.conf.HiveConf"); return true; } catch (ClassNotFoundException | NoClassDefFoundError e) { return false; } } {code} since "org.apache.spark.sql.hive.HiveSharedState" has been removed from spark2.1.x So hiveClassesArePresent will always return false. This will cause spark session with hive support would be created with "spark.sql.catalogImplementation=in-memory" May be just remove {code} this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState"); {code} is OK? -- This message was sent by Atlassian JIRA (v6.3.4#6332)