Hi team,
I was trying to query hive tables using Apache drill web interface. Was
facing following issues.
I am able to list tables and schemas. but while trying to query the table
its not able to execute.
Please find all the logs and snapshots from web ui below.
Stack Version :
Drill : 1.17.0
Hadoop 3.1.1.3.1.4.0-315
Hive 3.1.0.3.1.4.0-315
JAVA openjdk version "1.8.0_222"
Hive Storage Pluign :
{
"type": "hive",
"configProps": {
"hive.metastore.uris": "thrift://xx.xx.xx.xx:9083",
"hive.metastore.warehouse.dir": "/warehouse/tablespace/managed/hive",
"fs.default.name": "hdfs://xx.xx.xx.xx:8020",
"hive.metastore.sasl.enabled": "true",
"hive.server2.enable.doAs": "false",
"hive.metastore.execute.setugi": "true",
"hive.metastore.kerberos.principal": "hive/_HOST@<REALM>.COM",
"hive.security.authorization.enabled": "true"
},
"enabled": true
}
Drill Query Query :
select * from hiveasg.nifi_test1.assoc_5879238342_avro
Contents of drillbit.log file :
https://drive.google.com/file/d/1319oYbwFzprMQsGZ4BTTBT08YyiSWbTT
Contents of drillbit_queris.json :
{"queryId":"21d81830-2eab-0d43-f76e-1e021cd1e111","schema":"","queryText":"select
* from
hiveasg.nifi_test1.assoc_5879238342_avro","start":1579673551875,"finish":1579673551954,"outcome":"FAILED","username":"drill","remoteAddress":"xx.xx.xx.xx:48503"}
Web api Error snapshot :
https://drive.google.com/open?id=1Jt5h7do81bFhxtav9Y_W-6_bWKxArp2f
2020-01-22 06:12:31,878 [21d81830-2eab-0d43-f76e-1e021cd1e111:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query with id 21d81830-2eab-0d43-f76e-1e021cd1e111 issued by drill: select * from hiveasg.nifi_test1.assoc_5879238342_avro
2020-01-22 06:12:31,951 [21d81830-2eab-0d43-f76e-1e021cd1e111:foreman] INFO o.a.d.exec.planner.sql.SqlConverter - User Error Occurred: Failed to use the Hive authorization components: Unsupported privilege type CREATE (Failed to use the Hive authorization components: Unsupported privilege type CREATE)
org.apache.drill.common.exceptions.UserException: VALIDATION ERROR: Failed to use the Hive authorization components: Unsupported privilege type CREATE
[Error Id: 7ca9f828-d6bd-4613-84af-69ec710b5021 ]
at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:637)
at org.apache.drill.exec.planner.sql.SqlConverter.validate(SqlConverter.java:225)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:665)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:199)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:172)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:282)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:162)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:127)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:92)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:590)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:275)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failed to use the Hive authorization components: Unsupported privilege type CREATE
at org.apache.drill.exec.store.hive.client.HiveAuthorizationHelper.authorize(HiveAuthorizationHelper.java:153)
at org.apache.drill.exec.store.hive.client.HiveAuthorizationHelper.authorizeReadTable(HiveAuthorizationHelper.java:138)
at org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClientWithAuthorization.getHiveReadEntry(DrillHiveMetaStoreClientWithAuthorization.java:76)
at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getSelectionBaseOnName(HiveSchemaFactory.java:232)
at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getDrillTable(HiveSchemaFactory.java:213)
at org.apache.drill.exec.store.hive.schema.HiveDatabaseSchema.getTable(HiveDatabaseSchema.java:58)
at org.apache.calcite.jdbc.SimpleCalciteSchema.getImplicitTable(SimpleCalciteSchema.java:83)
at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:289)
at org.apache.calcite.sql.validate.EmptyScope.resolve_(EmptyScope.java:143)
at org.apache.calcite.sql.validate.EmptyScope.resolveTable(EmptyScope.java:99)
at org.apache.calcite.sql.validate.DelegatingScope.resolveTable(DelegatingScope.java:203)
at org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl(IdentifierNamespace.java:105)
at org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:177)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:995)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:955)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3109)
at org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom(SqlConverter.java:298)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3091)
at org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom(SqlConverter.java:298)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3363)
at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:995)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:955)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:930)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:637)
at org.apache.drill.exec.planner.sql.SqlConverter.validate(SqlConverter.java:218)
... 12 common frames omitted
Caused by: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: Unsupported privilege type CREATE
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.getRequirePrivilege(SQLPrivilegeType.java:37)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivTypeGrant.getSQLPrivTypeGrant(SQLPrivTypeGrant.java:79)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.RequiredPrivileges.addPrivilege(RequiredPrivileges.java:39)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.addRequiredPrivs(SQLAuthorizationUtils.java:353)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getRequiredPrivsFromThrift(SQLAuthorizationUtils.java:330)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromMetaStore(SQLAuthorizationUtils.java:209)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizationValidator.checkPrivileges(SQLStdHiveAuthorizationValidator.java:131)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizationValidator.checkPrivileges(SQLStdHiveAuthorizationValidator.java:83)
at org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl.checkPrivileges(HiveAuthorizerImpl.java:87)
at org.apache.drill.exec.store.hive.client.HiveAuthorizationHelper.authorize(HiveAuthorizationHelper.java:149)
... 40 common frames omitted
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.CREATE
at java.lang.Enum.valueOf(Enum.java:238)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.valueOf(SQLPrivilegeType.java:24)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.getRequirePrivilege(SQLPrivilegeType.java:35)
... 49 common frames omitted
2020-01-22 06:12:31,956 [qtp1349966375-61] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed: {}
org.apache.drill.common.exceptions.UserRemoteException: VALIDATION ERROR: Failed to use the Hive authorization components: Unsupported privilege type CREATE
[Error Id: 7ca9f828-d6bd-4613-84af-69ec710b5021 ]
at org.apache.drill.exec.server.rest.QueryWrapper.run(QueryWrapper.java:127)
at org.apache.drill.exec.server.rest.QueryResources.submitQueryJSON(QueryResources.java:86)
at org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
at org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(CsrfTokenValidateFilter.java:55)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(CsrfTokenInjectFilter.java:54)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:151)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failed to use the Hive authorization components: Unsupported privilege type CREATE
at org.apache.drill.exec.store.hive.client.HiveAuthorizationHelper.authorize(HiveAuthorizationHelper.java:153)
at org.apache.drill.exec.store.hive.client.HiveAuthorizationHelper.authorizeReadTable(HiveAuthorizationHelper.java:138)
at org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClientWithAuthorization.getHiveReadEntry(DrillHiveMetaStoreClientWithAuthorization.java:76)
at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getSelectionBaseOnName(HiveSchemaFactory.java:232)
at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getDrillTable(HiveSchemaFactory.java:213)
at org.apache.drill.exec.store.hive.schema.HiveDatabaseSchema.getTable(HiveDatabaseSchema.java:58)
at org.apache.calcite.jdbc.SimpleCalciteSchema.getImplicitTable(SimpleCalciteSchema.java:83)
at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:289)
at org.apache.calcite.sql.validate.EmptyScope.resolve_(EmptyScope.java:143)
at org.apache.calcite.sql.validate.EmptyScope.resolveTable(EmptyScope.java:99)
at org.apache.calcite.sql.validate.DelegatingScope.resolveTable(DelegatingScope.java:203)
at org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl(IdentifierNamespace.java:105)
at org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:177)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:995)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:955)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3109)
at org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom(SqlConverter.java:298)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3091)
at org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom(SqlConverter.java:298)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3363)
at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:995)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:955)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:930)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:637)
at org.apache.drill.exec.planner.sql.SqlConverter.validate(SqlConverter.java:218)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:665)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:199)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:172)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:282)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:162)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:127)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:92)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:590)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:275)
at .......(:0)
Caused by: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: Unsupported privilege type CREATE
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.getRequirePrivilege(SQLPrivilegeType.java:37)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivTypeGrant.getSQLPrivTypeGrant(SQLPrivTypeGrant.java:79)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.RequiredPrivileges.addPrivilege(RequiredPrivileges.java:39)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.addRequiredPrivs(SQLAuthorizationUtils.java:353)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getRequiredPrivsFromThrift(SQLAuthorizationUtils.java:330)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromMetaStore(SQLAuthorizationUtils.java:209)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizationValidator.checkPrivileges(SQLStdHiveAuthorizationValidator.java:131)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizationValidator.checkPrivileges(SQLStdHiveAuthorizationValidator.java:83)
at org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl.checkPrivileges(HiveAuthorizerImpl.java:87)
at org.apache.drill.exec.store.hive.client.HiveAuthorizationHelper.authorize(HiveAuthorizationHelper.java:149)
... 38 common frames omitted
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.CREATE
at .......(:0)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.valueOf(SQLPrivilegeType.java:24)
at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLPrivilegeType.getRequirePrivilege(SQLPrivilegeType.java:35)
... 47 common frames omitted
