Hi Anup,

As Sorabh mentioned, you seem to be using hive 2.1.1 jars in Drill classpath 
based on the stack trace. Did you build drill package by customizing hive 
version to 2.1.1 or added hive 2.1.1 jars in Drill classpath manually?  I could 
see that Drill 1.12 (latest released), 1.11  and 1.10 are bundled with hive 
1.2.1 client.


You could try with drill 1.11 tarball that includes hive 1.2.1 client. Also, 
you may check the version of hive client jars in $DRILL_HOME/jars/3rdparty/ 
directory from the Drill 1.10 cluster where it is working fine.


http://archive.apache.org/dist/drill/drill-1.11.0/


# List of hive client jars from Drill 1.11 tarball extracted.


$ ls -1 apache-drill-1.11.0/jars/3rdparty/hive-*
apache-drill-1.11.0/jars/3rdparty/hive-contrib-1.2.1.jar
apache-drill-1.11.0/jars/3rdparty/hive-hbase-handler-1.2.1.jar
apache-drill-1.11.0/jars/3rdparty/hive-metastore-1.2.1.jar
$


Thanks,


Arjun


________________________________
From: Anup Tiwari <anup.tiw...@games24x7.com>
Sent: Tuesday, February 13, 2018 12:01 PM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0

Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of
clusters and it is working properly. Just the change is in Hadoop Version and we
are using MySQL community version 5.7.16. Please find below details :-
Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache
Hadoop : 2.7.3
So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1
related jars in 3rdparty jar of drill.





On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?





On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwa...@mapr.com  wrote:
Hi Anup,




>From the stack trace which you have shared it looks like you are trying to use
Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.







Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.







java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:




java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)







Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.







[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[https://avatars3.githubusercontent.com/u/47359?s=400&v=4]<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>

apache/drill<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>
github.com
drill - Mirror of Apache Drill







[2]: https://github.com/apache/drill/pull/1111







Thanks,

Sorabh







________________________________

From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Monday, February 12, 2018 9:21 AM

To: user@drill.apache.org

Subject: Re: Unable to setup hive plugin in Drill 1.11.0




I don't know where to look for logs as i have tried multiple time creating

hive plugin. But one time i can see below stack is printed in drillbit.log(i

don't know if it came due to plugin creation or not as i have tried multiple

time but nothing got printed in drillbit.log except below which is present in

logs only once) :-

Error

org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up

new storage plugin configuration for config org.apache.drill.exec.

store.hive.HiveStoragePluginConfig@f149f108 at

org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)

[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)

[drill-java-exec-1.11.0.jar:1.11.0] at

sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]

  at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

~[na:1.8.0_72] at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)

~[na:1.8.0_72] at

org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java

:81) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)

  [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher

Provider.java:195) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104

) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.internal.Errors.process(Errors.java:315)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.internal.Errors.process(Errors.java:297)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.internal.Errors.process(Errors.java:267)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)

[jersey-container-servlet-core-2.8.jar:na] at

org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)

[jersey-container-servlet-core-2.8.jar:na] at

org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)

[jersey-container-servlet-core-2.8.jar:na] at

org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)

[jersey-container-servlet-core-2.8.jar:na] at

org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)

[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)

[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)

[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.Server.handle(Server.java:462)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)

[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)

[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)

[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at

org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)

[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at

java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:

java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at

org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)

~[hive-metastore-2.1.1.jar:2.1.1] at

org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)

~[hive-metastore-2.1.1.jar:2.1.1] at

org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)

~[drill-storage-hive-core-1.11.0.jar:1.11.0] at

sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

~[na:1.8.0_72] at

sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

~[na:1.8.0_72] at

sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

~[na:1.8.0_72] at

java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]

  at

org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)

~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
















On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiw...@games24x7.com wrote:

Hi All,

Please find below information :-

Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1

mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar

Hive Plugin info :-

{ "type" : "hive", "enabled" : true, "configProps" : {

  "hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",

  "hive.metastore.sasl.enabled" : "false", "fs.default.name" :

"hdfs://prod-hadoop-xxx:9000" } }




Error :

"result" : "error (unable to create/ update storage)"

Regards,

Anup Tiwari




Sent with Mixmax




Regards,

Anup Tiwari






Regards,
Anup Tiwari


Regards,
Anup Tiwari

Reply via email to