[jira] [Commented] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore
[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293307#comment-16293307 ] Marcelo Vanzin commented on SPARK-19804: Spark still doesn't have explicit support for Hive 2.2. > HiveClientImpl does not work with Hive 2.2.0 metastore > -- > > Key: SPARK-19804 > URL: https://issues.apache.org/jira/browse/SPARK-19804 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Marcelo Vanzin >Assignee: Marcelo Vanzin >Priority: Minor > Fix For: 2.2.0 > > > I know that Spark currently does not officially support Hive 2.2 (perhaps > because it hasn't been released yet); but we have some 2.2 patches in CDH and > the current code in the isolated client fails. The most probably culprit are > changes added in HIVE-13149. > The fix is simple, and here's the patch we applied in CDH: > https://github.com/cloudera/spark/commit/954f060afe6ed469e85d656abd02790a79ec07a0 > Fixing that doesn't affect any existing Hive version support, but will make > it easier to support 2.2 when it's out. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore
[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293302#comment-16293302 ] Zhongting Hu commented on SPARK-19804: -- [~smilegator] , I've made some comments on above PR, so for this issue, does it mean, before this fix (spark 2.2.0 release), all the version of spark can not talk to hive 2.2.0 metastore? > HiveClientImpl does not work with Hive 2.2.0 metastore > -- > > Key: SPARK-19804 > URL: https://issues.apache.org/jira/browse/SPARK-19804 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Marcelo Vanzin >Assignee: Marcelo Vanzin >Priority: Minor > Fix For: 2.2.0 > > > I know that Spark currently does not officially support Hive 2.2 (perhaps > because it hasn't been released yet); but we have some 2.2 patches in CDH and > the current code in the isolated client fails. The most probably culprit are > changes added in HIVE-13149. > The fix is simple, and here's the patch we applied in CDH: > https://github.com/cloudera/spark/commit/954f060afe6ed469e85d656abd02790a79ec07a0 > Fixing that doesn't affect any existing Hive version support, but will make > it easier to support 2.2 when it's out. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore
[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895459#comment-15895459 ] Xiao Li commented on SPARK-19804: - https://github.com/apache/spark/pull/17154 > HiveClientImpl does not work with Hive 2.2.0 metastore > -- > > Key: SPARK-19804 > URL: https://issues.apache.org/jira/browse/SPARK-19804 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Marcelo Vanzin >Priority: Minor > > I know that Spark currently does not officially support Hive 2.2 (perhaps > because it hasn't been released yet); but we have some 2.2 patches in CDH and > the current code in the isolated client fails. The most probably culprit are > changes added in HIVE-13149. > The fix is simple, and here's the patch we applied in CDH: > https://github.com/cloudera/spark/commit/954f060afe6ed469e85d656abd02790a79ec07a0 > Fixing that doesn't affect any existing Hive version support, but will make > it easier to support 2.2 when it's out. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore
[
https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893579#comment-15893579
]
Marcelo Vanzin commented on SPARK-19804:
For posterity, the error you get looks like this:
{noformat}
java.lang.ExceptionInInitializerError: null
at java.lang.Class.getConstructor0(Class.java:2892)
at java.lang.Class.getDeclaredConstructor(Class.java:2058)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1541)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3220)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3239)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3464)
at
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:226)
at
org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:210)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:294)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:269)
at
org.apache.spark.sql.hive.client.ClientWrapper.client(ClientWrapper.scala:272)
{noformat}
Which is rather cryptic; it's caused by one of the classes in the constructor
being loaded by two different class loaders, so {{getDeclaredConstructor}}
fails to find the right constructor and returns null.
> HiveClientImpl does not work with Hive 2.2.0 metastore
> --
>
> Key: SPARK-19804
> URL: https://issues.apache.org/jira/browse/SPARK-19804
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.2.0
>Reporter: Marcelo Vanzin
>Priority: Minor
>
> I know that Spark currently does not officially support Hive 2.2 (perhaps
> because it hasn't been released yet); but we have some 2.2 patches in CDH and
> the current code in the isolated client fails. The most probably culprit are
> changes added in HIVE-13149.
> The fix is simple, and here's the patch we applied in CDH:
> https://github.com/cloudera/spark/commit/954f060afe6ed469e85d656abd02790a79ec07a0
> Fixing that doesn't affect any existing Hive version support, but will make
> it easier to support 2.2 when it's out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
