Hi Arjun,
You have pointed correctly; i have removed all hive-2.1 jar from 3rd party and
kept only 3 jars which come with tar ball and it worked.
Thanks!!




On Tue, Feb 13, 2018 1:33 PM, Arjun kr arjun...@outlook.com  wrote:
Hi Anup,







As Sorabh mentioned, you seem to be using hive 2.1.1 jars in Drill classpath
based on the stack trace. Did you build drill package by customizing hive
version to 2.1.1 or added hive 2.1.1 jars in Drill classpath manually? I could
see that Drill 1.12 (latest released), 1.11 and 1.10 are bundled with hive 1.2.1
client.







You could try with drill 1.11 tarball that includes hive 1.2.1 client. Also, you
may check the version of hive client jars in $DRILL_HOME/jars/3rdparty/
directory from the Drill 1.10 cluster where it is working fine.







http://archive.apache.org/dist/drill/drill-1.11.0/







# List of hive client jars from Drill 1.11 tarball extracted.







$ ls -1 apache-drill-1.11.0/jars/3rdparty/hive-*

apache-drill-1.11.0/jars/3rdparty/hive-contrib-1.2.1.jar

apache-drill-1.11.0/jars/3rdparty/hive-hbase-handler-1.2.1.jar

apache-drill-1.11.0/jars/3rdparty/hive-metastore-1.2.1.jar

$







Thanks,







Arjun







________________________________

From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Tuesday, February 13, 2018 12:01 PM

To: user@drill.apache.org

Subject: Re: Unable to setup hive plugin in Drill 1.11.0




Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of

clusters and it is working properly. Just the change is in Hadoop Version and we

are using MySQL community version 5.7.16. Please find below details :-

Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache

Hadoop : 2.7.3

So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1

related jars in 3rdparty jar of drill.
















On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiw...@games24x7.com wrote:

Hi Sorabh,

Thanks for reply. We are using below combination :-

Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1

Apache Hadoop : 2.8.0

As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported

so do we have any link from where i can get an idea of which combination is

supported? Also does not supported means we can't use this combination(i mean

don't we have any workaround)?
















On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwa...@mapr.com wrote:

Hi Anup,













From the stack trace which you have shared it looks like you are trying to use

Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a

PR[2] open for it in 1.13 though.






















Also from the stack trace it's pointing towards an invalid field in HiveConf

METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.






















java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:













java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at













org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)













~[hive-metastore-2.1.1.jar:2.1.1] at













org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)













~[hive-metastore-2.1.1.jar:2.1.1] at













org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)






















Usually any error/exxception should get logged into drillbit.out as well. May be

you can try seeing there.






















[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51

[https://avatars3.githubusercontent.com/u/47359?s=400&v=4]<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>




apache/drill<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>

github.com

drill - Mirror of Apache Drill






















[2]: https://github.com/apache/drill/pull/1111






















Thanks,




Sorabh






















________________________________




From: Anup Tiwari <anup.tiw...@games24x7.com>




Sent: Monday, February 12, 2018 9:21 AM




To: user@drill.apache.org




Subject: Re: Unable to setup hive plugin in Drill 1.11.0













I don't know where to look for logs as i have tried multiple time creating




hive plugin. But one time i can see below stack is printed in drillbit.log(i




don't know if it came due to plugin creation or not as i have tried multiple




time but nothing got printed in drillbit.log except below which is present in




logs only once) :-




Error




org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up




new storage plugin configuration for config org.apache.drill.exec.




store.hive.HiveStoragePluginConfig@f149f108 at




org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)




~[drill-java-exec-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)




~[drill-java-exec-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)




~[drill-java-exec-1.11.0.jar:1.11.0] at




org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)




[drill-java-exec-1.11.0.jar:1.11.0] at




org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)




[drill-java-exec-1.11.0.jar:1.11.0] at




sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]




 at




sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)




~[na:1.8.0_72] at




sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)




~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)




~[na:1.8.0_72] at




org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java




:81) [jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)




 [jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher




Provider.java:195) [jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104




) [jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)




[jersey-common-2.8.jar:na] at




org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)




[jersey-common-2.8.jar:na] at




org.glassfish.jersey.internal.Errors.process(Errors.java:315)




[jersey-common-2.8.jar:na] at




org.glassfish.jersey.internal.Errors.process(Errors.java:297)




[jersey-common-2.8.jar:na] at




org.glassfish.jersey.internal.Errors.process(Errors.java:267)




[jersey-common-2.8.jar:na] at




org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)




[jersey-common-2.8.jar:na] at




org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)




[jersey-server-2.8.jar:na] at




org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)




[jersey-container-servlet-core-2.8.jar:na] at




org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)




[jersey-container-servlet-core-2.8.jar:na] at




org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)




[jersey-container-servlet-core-2.8.jar:na] at




org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)




[jersey-container-servlet-core-2.8.jar:na] at




org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)




[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)




[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)




[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.Server.handle(Server.java:462)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)




[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)




[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)




[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at




org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)




[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at




java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:




java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)




~[drill-storage-hive-core-1.11.0.jar:1.11.0] at




sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)




~[na:1.8.0_72] at




sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)




~[na:1.8.0_72] at




sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)




~[na:1.8.0_72] at




java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]




 at




org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)




~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted

















































On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiw...@games24x7.com wrote:




Hi All,




Please find below information :-




Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1




mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar




Hive Plugin info :-




{ "type" : "hive", "enabled" : true, "configProps" : {




 "hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",




 "hive.metastore.sasl.enabled" : "false", "fs.default.name" :




"hdfs://prod-hadoop-xxx:9000" } }













Error :




"result" : "error (unable to create/ update storage)"




Regards,




Anup Tiwari













Sent with Mixmax













Regards,




Anup Tiwari



















Regards,

Anup Tiwari







Regards,

Anup Tiwari





Regards,
Anup Tiwari

Sent with Mixmax

Reply via email to