Hi all,
I am trying to use drill to query a kerberized cluster. I have setup drill with
the provided in documentation for kerberized cluster.
the storage plugin details i am trying to create for hive is :
{
"type": "hive",
"configProps": {
"hive.metastore.uris":
"thrift://xxxx-xxx-xxxx:9083,thrift://xxxx-xxx-xxxx:9083",
"hive.metastore.warehouse.dir": "/warehouse/tablespace/managed/hive",
"fs.default.name": "hdfs://xxxx-xxx-xxxx:8020",
"hive.security.authorization.enabled": "true",
"hive.security.authenticator.manager":
"org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator",
"hive.security.authorization.manager":
"org.apcahe.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory",
"hive.metastore.sasl.enabled": "true",
"hive.server2.enable.doAs": "true"
},
"enabled": true
}
i am facing the following issue while trying to create hive storage plugin:
Please retry: Error while creating / updating storage : Could not connect to
meta store using any of the URIs provided.
Most recent failure: org.apache.thrift.transport.TTransportException:
GSS initiate failed at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at
org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:480)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:247)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:129)
at
org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:54)
at
org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClientFactory.createCloseableClientWithCaching(DrillHiveMetaStoreClientFactory.java:101)
at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:77)
at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:77)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:466)
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:131)
at
org.apache.drill.exec.server.rest.PluginConfigWrapper.createOrUpdateInStorage(PluginConfigWrapper.java:56)
at
org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePluginJSON(
StorageResources.java:193) at
org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugin(StorageResources.java:210)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
java.lang.reflect.Method.invoke(Method.java:498) at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at
org.glassfish.jersey.internal.Errors.process(Errors.java:315) at
org.glassfish.jersey.internal.Errors.process(Errors.java:297) at
org.glassfish.jersey.internal.Errors.process(Errors.java:267) at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
at \
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:151)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513) at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539) at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at
java.lang.Thread.run(Thread.java:748)
am i providing any wrong configurations while creating the storage plugin?
I have created drill keytab using the steps provided in the documentation and
have a mit kerberos autorization service.
let me know any solutions that can be used to solve this.
Thanks and Regards
Rameshwar Mane
Big Data Engineer
