也可能是需要这个fix：https://issues.apache.org/jira/browse/FLINK-20913
On Mon, Mar 8, 2021 at 12:11 PM Rui Li <[email protected]> wrote: > Hi， > > > 从你发的stacktrace来看，走到了set_ugi方法说明client认为server没有开启kerberos。确认一下你HiveCatalog这边指定的hive-site.xml是否配置正确呢，像hive.metastore.sasl.enabled是不是设置成true了？ > > On Sun, Mar 7, 2021 at 5:49 PM [email protected] <[email protected]> wrote: > >> 环境： >> flink1.12.1 >> hive2.1.0 >> CDH6.2.0 >> >> >> 【问题描述】 >> 在没开启Kerberos认证时，可以正常读写hive表 >> >> 开启Kerberos认证后， >> 启动时可以正常读取到hive metastore的元数据信息，读写不了表。 >> >> >> 【sql-client.sh embedded】 >> Flink SQL> show tables; >> dimension_table >> dimension_table1 >> test >> >> >> Flink SQL> select * from test; >> [ERROR] Could not execute SQL statement. Reason: >> org.apache.flink.connectors.hive.FlinkHiveException: Failed to collect >> all partitions from hive metaStore >> >> >> 【完整日志 >> /opt/cloudera/parcels/FLINK-1.12.1-BIN-SCALA_2.11/lib/flink/log/flink-root-sql-client-cdh6.com.log】 >> >> 2021-03-07 10:29:18.776 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Trying to connect to localhost/127.0.0.1:6123 >> 2021-03-07 10:29:18.777 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address 'cdh6.com/192.168.31.10': Connection >> refused (Connection refused) >> 2021-03-07 10:29:18.778 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:18.778 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/fe80:0:0:0:20c:29ff:fea1:6d6b%ens33': >> Network is unreachable (connect failed) >> 2021-03-07 10:29:18.778 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/192.168.31.10': Connection refused >> (Connection refused) >> 2021-03-07 10:29:18.779 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network is >> unreachable (connect failed) >> 2021-03-07 10:29:18.779 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:18.779 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/fe80:0:0:0:20c:29ff:fea1:6d6b%ens33': >> Network is unreachable (connect failed) >> 2021-03-07 10:29:18.779 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/192.168.31.10': Connection refused >> (Connection refused) >> 2021-03-07 10:29:18.780 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network is >> unreachable (connect failed) >> 2021-03-07 10:29:18.780 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:18.780 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Could not connect. Waiting for 1600 msecs before next attempt >> 2021-03-07 10:29:20.381 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Trying to connect to localhost/127.0.0.1:6123 >> 2021-03-07 10:29:20.381 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address 'cdh6.com/192.168.31.10': Connection >> refused (Connection refused) >> 2021-03-07 10:29:20.382 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:20.383 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/fe80:0:0:0:20c:29ff:fea1:6d6b%ens33': >> Network is unreachable (connect failed) >> 2021-03-07 10:29:20.383 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/192.168.31.10': Connection refused >> (Connection refused) >> 2021-03-07 10:29:20.383 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network is >> unreachable (connect failed) >> 2021-03-07 10:29:20.383 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:20.384 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/fe80:0:0:0:20c:29ff:fea1:6d6b%ens33': >> Network is unreachable (connect failed) >> 2021-03-07 10:29:20.384 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/192.168.31.10': Connection refused >> (Connection refused) >> 2021-03-07 10:29:20.384 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network is >> unreachable (connect failed) >> 2021-03-07 10:29:20.385 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:20.385 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Could not connect. Waiting for 1829 msecs before next attempt >> 2021-03-07 10:29:22.214 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Trying to connect to localhost/127.0.0.1:6123 >> 2021-03-07 10:29:22.215 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address 'cdh6.com/192.168.31.10': Connection >> refused (Connection refused) >> 2021-03-07 10:29:22.216 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:22.216 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/fe80:0:0:0:20c:29ff:fea1:6d6b%ens33': >> Network is unreachable (connect failed) >> 2021-03-07 10:29:22.217 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/192.168.31.10': Connection refused >> (Connection refused) >> 2021-03-07 10:29:22.217 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network is >> unreachable (connect failed) >> 2021-03-07 10:29:22.217 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:22.218 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/fe80:0:0:0:20c:29ff:fea1:6d6b%ens33': >> Network is unreachable (connect failed) >> 2021-03-07 10:29:22.218 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/192.168.31.10': Connection refused >> (Connection refused) >> 2021-03-07 10:29:22.218 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network is >> unreachable (connect failed) >> 2021-03-07 10:29:22.218 [main] INFO >> org.apache.flink.runtime.net.ConnectionUtils >> - Failed to connect from address '/127.0.0.1': Connection refused >> (Connection refused) >> 2021-03-07 10:29:22.219 [main] WARN >> org.apache.flink.runtime.net.ConnectionUtils >> - Could not connect to localhost/127.0.0.1:6123. Selecting a local >> address using heuristics. >> 2021-03-07 10:29:22.290 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.vectorized.use.checked.expressions does not exist >> 2021-03-07 10:29:22.290 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.vectorized.use.checked.expressions does not exist >> 2021-03-07 10:29:22.290 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.strict.checks.no.partition.filter does not exist >> 2021-03-07 10:29:22.290 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.strict.checks.no.partition.filter does not exist >> 2021-03-07 10:29:22.290 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.strict.checks.orderby.no.limit does not exist >> 2021-03-07 10:29:22.290 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.strict.checks.orderby.no.limit does not exist >> 2021-03-07 10:29:22.291 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.vectorized.input.format.excludes does not exist >> 2021-03-07 10:29:22.291 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.vectorized.input.format.excludes does not exist >> 2021-03-07 10:29:22.291 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.strict.checks.bucketing does not exist >> 2021-03-07 10:29:22.291 [main] WARN >> org.apache.hadoop.hive.conf.HiveConf - HiveConf of name >> hive.strict.checks.bucketing does not exist >> 2021-03-07 10:29:22.291 [main] INFO hive.metastore - Trying >> to connect to metastore with URI thrift://cdh6.com:9083 >> 2021-03-07 10:29:22.292 [main] INFO hive.metastore - Opened a >> connection to metastore, current connections: 2 >> 2021-03-07 10:29:22.301 [main] WARN hive.metastore - >> set_ugi() not successful, Likely cause: new client talking to old server. >> Continuing without it. >> org.apache.thrift.transport.TTransportException: null >> at >> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) >> at >> org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230) >> at >> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) >> at >> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:4122) >> at >> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:4108) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:495) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:286) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:211) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at >> java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> at >> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1652) >> at >> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) >> at >> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) >> at >> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:118) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at >> java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.apache.flink.table.catalog.hive.client.HiveShimV200.getHiveMetastoreClient(HiveShimV200.java:54) >> at >> org.apache.flink.table.catalog.hive.client.HiveMetastoreClientWrapper.createMetastoreClient(HiveMetastoreClientWrapper.java:274) >> at >> org.apache.flink.table.catalog.hive.client.HiveMetastoreClientWrapper.<init>(HiveMetastoreClientWrapper.java:80) >> at >> org.apache.flink.table.catalog.hive.client.HiveMetastoreClientFactory.create(HiveMetastoreClientFactory.java:32) >> at >> org.apache.flink.connectors.hive.util.HivePartitionUtils.getAllPartitions(HivePartitionUtils.java:114) >> at >> org.apache.flink.connectors.hive.HiveTableSource.getDataStream(HiveTableSource.java:137) >> at >> org.apache.flink.connectors.hive.HiveTableSource$1.produceDataStream(HiveTableSource.java:122) >> at >> org.apache.flink.table.planner.plan.nodes.common.CommonPhysicalTableSourceScan.createSourceTransformation(CommonPhysicalTableSourceScan.scala:88) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecTableSourceScan.translateToPlanInternal(StreamExecTableSourceScan.scala:91) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecTableSourceScan.translateToPlanInternal(StreamExecTableSourceScan.scala:44) >> at >> org.apache.flink.table.planner.plan.nodes.exec.ExecNode$class.translateToPlan(ExecNode.scala:59) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecTableSourceScan.translateToPlan(StreamExecTableSourceScan.scala:44) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToTransformation(StreamExecLegacySink.scala:158) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToPlanInternal(StreamExecLegacySink.scala:82) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToPlanInternal(StreamExecLegacySink.scala:48) >> at >> org.apache.flink.table.planner.plan.nodes.exec.ExecNode$class.translateToPlan(ExecNode.scala:59) >> at >> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToPlan(StreamExecLegacySink.scala:48) >> at >> org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:66) >> at >> org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:65) >> at >> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) >> at >> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) >> at >> scala.collection.Iterator$class.foreach(Iterator.scala:891) >> at >> scala.collection.AbstractIterator.foreach(Iterator.scala:1334) >> at >> scala.collection.IterableLike$class.foreach(IterableLike.scala:72) >> at >> scala.collection.AbstractIterable.foreach(Iterable.scala:54) >> at >> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) >> at >> scala.collection.AbstractTraversable.map(Traversable.scala:104) >> at >> org.apache.flink.table.planner.delegation.StreamPlanner.translateToPlan(StreamPlanner.scala:65) >> at >> org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:167) >> at >> org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1329) >> at >> org.apache.flink.table.api.internal.TableEnvironmentImpl.translateAndClearBuffer(TableEnvironmentImpl.java:1321) >> at >> org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.getPipeline(StreamTableEnvironmentImpl.java:328) >> at >> org.apache.flink.table.client.gateway.local.ExecutionContext.lambda$createPipeline$1(ExecutionContext.java:287) >> at >> org.apache.flink.table.client.gateway.local.ExecutionContext.wrapClassLoader(ExecutionContext.java:256) >> at >> org.apache.flink.table.client.gateway.local.ExecutionContext.createPipeline(ExecutionContext.java:282) >> at >> org.apache.flink.table.client.gateway.local.LocalExecutor.executeQueryInternal(LocalExecutor.java:542) >> at >> org.apache.flink.table.client.gateway.local.LocalExecutor.executeQuery(LocalExecutor.java:374) >> at >> org.apache.flink.table.client.cli.CliClient.callSelect(CliClient.java:648) >> at >> org.apache.flink.table.client.cli.CliClient.callCommand(CliClient.java:323) >> at >> java.util.Optional.ifPresent(Optional.java:159) >> at >> org.apache.flink.table.client.cli.CliClient.open(CliClient.java:214) >> at >> org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:144) >> at >> org.apache.flink.table.client.SqlClient.start(SqlClient.java:115) >> at >> org.apache.flink.table.client.SqlClient.main(SqlClient.java:201) >> 2021-03-07 10:29:22.302 [main] INFO hive.metastore - >> Connected to metastore. >> >> >> >> [email protected] >> > > > -- > Best regards! > Rui Li > -- Best regards! Rui Li
