Something weird. Instead of using the hive keytab to insert data, I've found that if the base directory (/apps/hive/warehouse/test.db/mytab/base_0000013) belongs to the 'hive' user, then the compaction succeeds (cleaning step is ok). Even if the delta_ directories don't belong to hive. Weird, isn't it ? Thus, this is a workaround but a little bit crappy. But I'm open to any more suitable solution.
Le lun. 26 août 2019 à 08:51, David Morin <morin.david....@gmail.com> a écrit : > Sorry, the same link in english: > http://www.adaltas.com/en/2019/07/25/hive-3-features-tips-tricks/ > > Le lun. 26 août 2019 à 08:35, David Morin <morin.david....@gmail.com> a > écrit : > >> Here after a link related to hive3: >> http://www.adaltas.com/fr/2019/07/25/hive-3-fonctionnalites-conseils-astuces/ >> The author suggests the use of the hive user. So the hive keytab to >> execute merge query due to kerberos on my side. Is it the only way to make >> compaction work properly ? >> >> Le lun. 26 août 2019 à 07:51, David Morin <morin.david....@gmail.com> a >> écrit : >> >>> Hello, >>> I've been trying "ALTER TABLE (table_name) COMPACT 'MAJOR'" on my Hive 2 >>> environment, but it always fails (HDP 2.6.5 precisely). It seems that the >>> merged base file is created but the delta is not deleted. >>> I found that it was because the HiveMetastore Client can't connect to >>> the metastore because of GSS Initiate failed during cleaning step. >>> >>> Here after the logs on the Metastore side (DEBUG mode activated): >>> >>> 2019-08-25 21:42:51,729 INFO [Thread-24]: hive.metastore >>> *(HiveMetaStoreClient.java:open(451)) >>> - Trying to connect to metastore with URI thrift://XXX:9083* >>> 2019-08-25 21:42:51,747 DEBUG [Thread-24]: security.UserGroupInformation >>> *(UserGroupInformation.java:logPrivilegedAction(1896)) >>> - PrivilegedAction as:myuser (auth:PROXY) via hive/XXXX (auth:KERBEROS) * >>> from:org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) >>> 2019-08-25 21:42:51,748 DEBUG [Thread-24]: transport.TSaslTransport >>> (TSaslTransport.java:open(261)) - opening transport >>> org.apache.thrift.transport.TSaslClientTransport@58fdf6d4 >>> 2019-08-25 21:42:51,750 ERROR [Thread-24]: transport.TSaslTransport >>> (TSaslTransport.java:open(315)) - SASL negotiation failure >>> *javax.security.sasl.SaslException: GSS initiate failed [Caused by >>> GSSException: No valid credentials provided (Mechanism level: Failed to >>> find any Kerberos tgt)]* >>> at >>> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) >>> at >>> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) >>> at >>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) >>> at >>> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:422) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) >>> at >>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:495) >>> at >>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:290) >>> at >>> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) >>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>> at >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >>> at >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) >>> at >>> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) >>> at >>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) >>> at >>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) >>> at >>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) >>> at >>> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3641) >>> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3675) >>> at >>> org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3917) >>> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225) >>> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:217) >>> at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner.removeFiles(Cleaner.java:344) >>> at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner.access$000(Cleaner.java:63) >>> at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner$1.run(Cleaner.java:295) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:422) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) >>> at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner.clean(Cleaner.java:292) >>> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:162) >>> Caused by: GSSException: No valid credentials provided (Mechanism level: >>> Failed to find any Kerberos tgt) >>> at >>> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) >>> at >>> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) >>> at >>> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) >>> at >>> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) >>> at >>> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) >>> at >>> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) >>> at >>> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) >>> ... 33 more >>> 2019-08-25 21:42:51,763 DEBUG [Thread-24]: transport.TSaslTransport >>> (TSaslTransport.java:sendSaslMessage(162)) - CLIENT: Writing message with >>> status BAD and payload length 19 >>> 2019-08-25 21:42:51,764 WARN [Thread-24]: hive.metastore >>> *(HiveMetaStoreClient.java:open(505)) - Failed to connect to the >>> MetaStore Server...org.apache.thrift.transport.TTransportException: GSS >>> initiate failed* >>> at >>> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) >>> at >>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) >>> at >>> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:422) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) >>> at >>> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) >>> at >>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:495) >>> at >>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:290) >>> at >>> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) >>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>> at >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >>> at >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) >>> at >>> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) >>> at >>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) >>> at >>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) >>> at >>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) >>> at >>> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3641) >>> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3675) >>> at >>> org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3917) >>> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225) >>> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:217) >>> >>> >>> * at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner.removeFiles(Cleaner.java:344) >>> at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner.access$000(Cleaner.java:63) >>> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner$1.run(Cleaner.java:295)* >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:422) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) >>> at >>> org.apache.hadoop.hive.ql.txn.compactor.Cleaner.clean(Cleaner.java:292) >>> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:162) >>> 2019-08-25 21:42:51,764 INFO [Thread-24]: hive.metastore >>> (HiveMetaStoreClient.java:open(539)) - Waiting 5 seconds before next >>> connection attempt. >>> 2019-08-25 21:42:51,766 DEBUG [pool-7-thread-1]: >>> security.UserGroupInformation >>> *(UserGroupInformation.java:logPrivilegedAction(1896)) >>> - PrivilegedAction as:hive/XXXXX (auth:KERBEROS) * >>> from:org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:606) >>> 2019-08-25 21:42:51,769 DEBUG [pool-7-thread-1]: >>> transport.TSaslServerTransport >>> (TSaslServerTransport.java:getTransport(213)) - transport map does not >>> contain key >>> 2019-08-25 21:42:51,769 DEBUG [pool-7-thread-1]: >>> transport.TSaslTransport (TSaslTransport.java:open(261)) - opening >>> transport org.apache.thrift.transport.TSaslServerTransport@327e3ff0 >>> 2019-08-25 21:42:51,771 DEBUG [pool-7-thread-1]: >>> transport.TSaslServerTransport >>> (TSaslServerTransport.java:getTransport(218)) - failed to open server >>> transport >>> org.apache.thrift.transport.TTransportException: Peer indicated failure: >>> GSS initiate failed >>> at >>> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199) >>> at >>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125) >>> at >>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) >>> at >>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) >>> at >>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) >>> at >>> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:609) >>> at >>> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:606) >>> at java.security.AccessController.doPrivileged(Native Method) >>> >>
