[
https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610286#comment-15610286
]
KaiXu commented on SPARK-18112:
-------------------------------
Spark 2.0 removes JavaSparkListener and change SparkListener from a trait to an
abstract class: https://issues.apache.org/jira/browse/SPARK-14358, but Hive2.x
still use JavaSparkListener: https://issues.apache.org/jira/browse/SPARK-17563,
this Hive patch solved this issue:
https://issues.apache.org/jira/browse/HIVE-14029, but from spark side it still
does not support read data from Hive2.x metastore.
> Spark2.x does not support read data from Hive 2.x metastore
> -----------------------------------------------------------
>
> Key: SPARK-18112
> URL: https://issues.apache.org/jira/browse/SPARK-18112
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit, SQL
> Affects Versions: 2.0.0, 2.0.1
> Reporter: KaiXu
>
> Hive2.0 has been released in February 2016, after that Hive2.0.1 and
> Hive2.1.0 have also been released for a long time, but till now spark only
> support to read hive metastore data from Hive1.2.1 and older version, since
> Hive2.x has many bugs fixed and performance improvement it's better and
> urgent to upgrade to support Hive2.x
> failed to load data from hive2.x metastore:
> Exception in thread "main" java.lang.NoSuchFieldError: HIVE_STATS_JDBC_TIMEOUT
> at
> org.apache.spark.sql.hive.HiveUtils$.hiveClientConfigurations(HiveUtils.scala:197)
> at
> org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:262)
> at
> org.apache.spark.sql.hive.HiveSharedState.metadataHive$lzycompute(HiveSharedState.scala:39)
> at
> org.apache.spark.sql.hive.HiveSharedState.metadataHive(HiveSharedState.scala:38)
> at
> org.apache.spark.sql.hive.HiveSharedState.externalCatalog$lzycompute(HiveSharedState.scala:4
> at
> org.apache.spark.sql.hive.HiveSharedState.externalCatalog(HiveSharedState.scala:45)
> at
> org.apache.spark.sql.hive.HiveSessionState.catalog$lzycompute(HiveSessionState.scala:50)
> at
> org.apache.spark.sql.hive.HiveSessionState.catalog(HiveSessionState.scala:48)
> at
> org.apache.spark.sql.hive.HiveSessionState.catalog(HiveSessionState.scala:31)
> at org.apache.spark.sql.SparkSession.table(SparkSession.scala:568)
> at org.apache.spark.sql.SparkSession.table(SparkSession.scala:564)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
