I did it! "KrbException: Clock skew too great (37) - PROCESS_TGS" means my
windows clock is not synchronized with the kerberos server clock.
After I do synchronized between windows and linux kerberos server. Every thing
goes well.
I am so grateful to you two.(^_^)
Maria.
At 2016-07-05 09:59:04, "Maria" <linanmengxia...@126.com> wrote:
>
>Yup,yesterday I started to realize that The renewal is a principal level
>setting. I hava fixed renew time in KDC kdc.conf. Do as Aviral said, I enable
>kerberos logs with
> "-Dsun.security.krb5.debug=true" , more error info printed out:
>------------------------------------------------------------------------------------------------
>Java config name: E:\Program Files (x86)\Java\jre7\lib\security\krb5.conf
>Loaded from Java config
>Java config name: E:\Program Files (x86)\Java\jre7\lib\security\krb5.conf
>Loaded from Java config
>>>> KdcAccessibility: reset
>>>> KdcAccessibility: reset
>>>> KeyTabInputStream, readName(): HADOOP.COM
>>>> KeyTabInputStream, readName(): hive
>>>> KeyTabInputStream, readName(): hm
>>>> KeyTab: load() entry length: 69; type: 18
>>>> KeyTabInputStream, readName(): HADOOP.COM
>>>> KeyTabInputStream, readName(): hive
>>>> KeyTabInputStream, readName(): hm
>>>> KeyTab: load() entry length: 53; type: 17
>>>> KeyTabInputStream, readName(): HADOOP.COM
>>>> KeyTabInputStream, readName(): hive
>>>> KeyTabInputStream, readName(): hm
>>>> KeyTab: load() entry length: 61; type: 16
>>>> KeyTabInputStream, readName(): HADOOP.COM
>>>> KeyTabInputStream, readName(): hive
>>>> KeyTabInputStream, readName(): hm
>>>> KeyTab: load() entry length: 53; type: 23
>>>> KeyTabInputStream, readName(): HADOOP.COM
>>>> KeyTabInputStream, readName(): hive
>>>> KeyTabInputStream, readName(): hm
>>>> KeyTab: load() entry length: 45; type: 8
>>>> KeyTabInputStream, readName(): HADOOP.COM
>>>> KeyTabInputStream, readName(): hive
>>>> KeyTabInputStream, readName(): hm
>>>> KeyTab: load() entry length: 45; type: 3
>Added key: 3version: 1
>Found unsupported keytype (8) for hive/h...@hadoop.com
>Added key: 23version: 1
>Added key: 16version: 1
>Added key: 17version: 1
>Found unsupported keytype (18) for hive/h...@hadoop.com
>Ordering keys wrt default_tkt_enctypes list
>Using builtin default etypes for default_tkt_enctypes
>default etypes for default_tkt_enctypes: 17 16 23 1 3.
>Added key: 3version: 1
>Found unsupported keytype (8) for hive/h...@hadoop.com
>Added key: 23version: 1
>Added key: 16version: 1
>Added key: 17version: 1
>Found unsupported keytype (18) for hive/h...@hadoop.com
>Ordering keys wrt default_tkt_enctypes list
>Using builtin default etypes for default_tkt_enctypes
>default etypes for default_tkt_enctypes: 17 16 23 1 3.
>Using builtin default etypes for default_tkt_enctypes
>default etypes for default_tkt_enctypes: 17 16 23 1 3.
>>>> KrbAsReq creating message
>>>> KrbKdcReq send: kdc=hm UDP:88, timeout=30000, number of retries =3,
>>>> #bytes=145
>>>> KDCCommunication: kdc=hm UDP:88, timeout=30000,Attempt =1, #bytes=145
>>>> KrbKdcReq send: #bytes read=598
>>>> KdcAccessibility: remove hm
>Added key: 3version: 1
>Found unsupported keytype (8) for hive/h...@hadoop.com
>Added key: 23version: 1
>Added key: 16version: 1
>Added key: 17version: 1
>Found unsupported keytype (18) for hive/h...@hadoop.com
>Ordering keys wrt default_tkt_enctypes list
>Using builtin default etypes for default_tkt_enctypes
>default etypes for default_tkt_enctypes: 17 16 23 1 3.
>>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
>>>> KrbAsRep cons in KrbAsReq.getReply hive/hm
>Added key: 3version: 1
>Found unsupported keytype (8) for hive/h...@hadoop.com
>Added key: 23version: 1
>Added key: 16version: 1
>Added key: 17version: 1
>Found unsupported keytype (18) for hive/h...@hadoop.com
>Ordering keys wrt default_tkt_enctypes list
>Using builtin default etypes for default_tkt_enctypes
>default etypes for default_tkt_enctypes: 17 16 23 1 3.
>start connect hiveserver..
>Found ticket for hive/h...@hadoop.com to go to krbtgt/hadoop....@hadoop.com
>expiring on Wed Jul 06 09:29:15 CST 2016
>Entered Krb5Context.initSecContext with state=STATE_NEW
>Found ticket for hive/h...@hadoop.com to go to krbtgt/hadoop....@hadoop.com
>expiring on Wed Jul 06 09:29:15 CST 2016
>Service ticket not found in the subject
>>>> Credentials acquireServiceCreds: same realm
>Using builtin default etypes for default_tgs_enctypes
>default etypes for default_tgs_enctypes: 17 16 23 1 3.
>>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
>>>> KrbKdcReq send: kdc=hm UDP:88, timeout=30000, number of retries =3,
>>>> #bytes=619
>>>> KDCCommunication: kdc=hm UDP:88, timeout=30000,Attempt =1, #bytes=619
>>>> KrbKdcReq send: #bytes read=116
>>>> KdcAccessibility: remove hm
>>>> KDCRep: init() encoding tag is 126 req type is 13
>>>>KRBError:
> cTime is Wed Jul 04 22:58:32 CST 1984 457801112000
> sTime is Tue Jul 05 09:29:15 CST 2016 1467682155000
> suSec is 944361
> error code is 37
> error Message is Clock skew too great
> realm is HADOOP.COM
> sname is hive/hm
> msgType is 30
>KrbException: Clock skew too great (37) - PROCESS_TGS
> at sun.security.krb5.KrbTgsRep.<init>(Unknown Source)
> at sun.security.krb5.KrbTgsReq.getReply(Unknown Source)
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(Unknown Source)
> at sun.security.krb5.internal.CredentialsUtil.serviceCreds(Unknown
> Source)
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(Unknown Source)
> at sun.security.krb5.Credentials.acquireServiceCreds(Unknown Source)
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Unknown Source)
> at sun.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
> at sun.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(Unknown Source)
> at
> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
> at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
> at java.sql.DriverManager.getConnection(Unknown Source)
> at java.sql.DriverManager.getConnection(Unknown Source)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.KerberosTest.main(KerberosTest.java:50)
>Caused by: KrbException: Identifier doesn't match expected value (906)
> at sun.security.krb5.internal.KDCRep.init(Unknown Source)
> at sun.security.krb5.internal.TGSRep.init(Unknown Source)
> at sun.security.krb5.internal.TGSRep.<init>(Unknown Source)
> ... 25 more
>java.sql.SQLException: Could not open client transport with JDBC Uri:
>jdbc:hive2://hm:10000/default;principal=hive/h...@hadoop.com: GSS initiate
>failed
> at
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:231)
> at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
> at java.sql.DriverManager.getConnection(Unknown Source)
> at java.sql.DriverManager.getConnection(Unknown Source)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.KerberosTest.main(KerberosTest.java:50)
>Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
> at
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
> ... 5 more
>
>As if kerberos configuration is incorrect ....
>
>
>At 2016-07-04 21:26:53, "Vivek Shrivastava" <vivshrivast...@gmail.com> wrote:
>
>
>The renewal lifetime at client krb5.conf level does make any difference. The
>renewal time period is defined at kdc in kdc.conf. Client can not override
>it. The renewal is also a property set at the principal level, both the
>settings ( renewal_lifetime, +renewal ) dictate if a ticket can be renewed. I
>don't think your problem has anything to do with that.
>
>
>Seems something basic is missing in your environment. I would probably, run
>the same piece of code in the unix environment and ensure that there is no
>error. Enabling Kerberos debugging logging as suggested in the previous post
>will also help you compare the sequence of execution.
>
>
>On Mon, Jul 4, 2016 at 7:52 AM, Aviral Agarwal <aviral12...@gmail.com> wrote:
>
>
>Hi,
>Could you enable kerberos logs with
>
> -Dsun.security.krb5.debug=true
>
>
>and paste the output ?
>
>
>
>
>On Mon, Jul 4, 2016 at 3:47 PM, Maria <linanmengxia...@126.com> wrote:
>
>The qestion "kinit: Ticket expired while renewing credentials" has been
>solved. I can successfully execute "kinit -R",
>
>but the error “java.lang.RuntimeException:
>org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS
>initiate failed”
>
>is still there..
>
>
>
>
>
>At 2016-07-04 14:39:04, "Maria" <linanmengxia...@126.com> wrote:
>
>>I saw a mail named "HCatalog Security",His or her problem was similar to
>>mine,and the reply answer were:
>
>>"This issue goes away after doing a kinit -R".
>
>>
>
>>So I did the same operation.while it is failed:
>
>>kinit: Ticket expired while renewing credentials
>
>>
>
>>But in my /etc/krb5.conf, I have configed this item:
>
>>renew_lifetime=7d
>
>>
>
>>So, Can anybody give me some suggestions, please? Thankyou.
>
>>
>
>>At 2016-07-04 11:32:30, "Maria" <linanmengxia...@126.com> wrote:
>
>>>
>
>>>
>
>>>And I can suucessfully access hiveserver2 from beeline.
>
>>>
>
>>>
>
>>>I was so confused by this error"Peer indicated failure: GSS initiate failed".
>
>>>
>
>>> Can you anybody please help me? Any reply will be much appreciated.
>
>>>
>
>>>At 2016-07-04 11:26:53, "Maria" <linanmengxia...@126.com> wrote:
>
>>>>Yup,my hiveserver2 log errors are:
>
>>>>
>
>>>>ERROR [Hiveserver2-Handler-Pool:
>>>>Thread-48]:server.TThreadPoolServer(TThreadPoolServer.java:run(296)) -
>>>>error occurred during processing of message.
>
>>>>java.lang.RuntimeException:
>>>>org.apache.thrift.transport.TTransportException: Peer indicated failure:
>>>>GSS initiate failed
>
>>>> at
>>>>org.apache.thrift.transport.TSaslServerTransport$FactorygetTransport(TSaslServerTransport.java:219)
>
>>>> at
>>>>org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:739)
>
>>>> at
>>>>org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:736)
>
>>>> at java.security.AccessController.doPrivileged(Native Method)
>
>>>> at javax.security.auth.Subject.doAs(Subject.java:356)
>
>>>> at
>>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
>
>>>> at
>>>>org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:736)
>
>>>> at
>>>>org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
>
>>>> at
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
>>>> at
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
>>>> at java.lang.Thread.run(Thread.java:745)
>
>>>>Caused by: org.apache.thrift.transport.TTransportException:Peer indicated
>>>>failure: GSS initiate failed
>
>>>> at
>>>>org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)
>
>>>> at
>>>>org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
>
>>>> at
>>>>org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
>
>>>> at
>>>>org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
>
>>>> at
>>>>org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>
>>>> ... 10 more
>
>>>>================================================
>
>>>>As if the windows hive JDBC client can communicate with the
>>>>hiveserver2,isn't it?
>
>>>>
>
>>>>while I checked everything I can :
>
>>>>(1)in hiveserver2 node, I execute command "klist",the results are:
>
>>>>Ticket cache: FILE:/tmp/krb5cc_0
>
>>>>Default principal: hive/h...@hadoop.com
>
>>>>
>
>>>>Valid starting Expires Service principal
>
>>>>07/04/16 10:28:14 07/05/16 10:28:14 krbtgt/hadoop....@hadoop.com
>
>>>> renew until 07/04/16 10:28:14
>
>>>>(2)in windows dos cmd,I execute command "klist",the results are:
>
>>>>Ticket cache:API: 1
>
>>>>Default principal: hive/h...@hadoop.com
>
>>>>
>
>>>>Valid starting Expires Service principal
>
>>>>07/04/16 10:24:32 07/05/16 10:24:32 krbtgt/hadoop....@hadoop.com
>
>>>> renew until 07/04/16 10:24:32
>
>>>>
>
>>>> Is there any thing else I have to add or set for hiveserver2?
>
>>>>
>
>>>>Thanks in advance.
>
>>>>
>
>>>>
>
>>>>Maria.
>
>>>>
>
>>>>At 2016-07-03 04:39:31, "Vivek Shrivastava" <vivshrivast...@gmail.com>
>>>>wrote:
>
>>>>
>
>>>>
>
>>>>Please look at the hiveserver2 log, it will have better error information.
>>>>You can paste error from the logs if you need help.
>
>>>>
>
>>>>
>
>>>>Regards,
>
>>>>
>
>>>>
>
>>>>Vivek
>
>>>>
>
>>>>
>
>>>>On Sat, Jul 2, 2016 at 5:52 AM, Maria <linanmengxia...@126.com> wrote:
>
>>>>
>
>>>>
>
>>>>
>
>>>>Hi,all:
>
>>>>
>
>>>> recently,I attempted to access Kerberized hadoop cluster by launching
>>>>JAVA applications from Windows workstations. And I hava configured kerberos
>>>>in my windows7, and can successfully access hdfs50070. But when I launch
>>>>JDBC from windows to connection remote hiveserver,errors accured:
>
>>>>
>
>>>>java.sql.SQLException:could not open client transport with JDBC
>>>>Uri:jdbc:hive2://hm:10000/default;principal=hive/h...@hadoom.com: GSS
>>>>initiate failed
>
>>>>
>
>>>> at
>>>>org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:231)
>
>>>>
>
>>>> at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176)
>
>>>>
>
>>>> at org.apache.hive.jdbc.HiveDriver.connection(HiveDriver.java:105)
>
>>>>
>
>>>> at java.sql.DriverManager.getConnection(Unknown Source)
>
>>>>
>
>>>> at java.sql.DriverManager.getConnection(Unknown Source)
>
>>>>
>
>>>> at
>>>>org.apache.hadoop.hive.ql.security.authorization.plugin.KerberosTest.main(KerberosTest.java:41)
>
>>>>
>
>>>>Caused by: org.apache.thrift.transport.TTransportException:GSS initiate
>>>>failed
>
>>>>
>
>>>> at
>>>>org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
>
>>>>
>
>>>> at
>>>>org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
>
>>>>
>
>>>> at
>>>>org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>
>>>>
>
>>>> at
>>>>org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
>
>>>>
>
>>>> at
>>>>org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
>
>>>>
>
>>>> at java.security.AccessController.doPrivileged(Native Method)
>
>>>>
>
>>>> at javax.security.auth.Subject.doAs(Unknow source)
>
>>>>
>
>>>> at
>>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>
>>>>
>
>>>> at
>>>>org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
>
>>>>
>
>>>> at
>>>>org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
>
>>>>
>
>>>>... 5 more
>
>>>>
>
>>>>------------------------------------------------------------------------------
>
>>>>
>
>>>>below are my test codes:
>
>>>>
>
>>>>
>
>>>>
>
>>>>public static void main(String[] args) {
>
>>>>
>
>>>> String principal = "hive/h...@hadoom.com";
>
>>>>
>
>>>> String keytab = "E:\\Program Files
>>>>(x86)\\java\\jre7\\lib\\security\\hive.keytab";
>
>>>>
>
>>>> String url =
>>>>"jdbc:hive2://hm:10000/default;principal=hive/h...@hadoom.com";
>
>>>>
>
>>>>
>
>>>>
>
>>>> conf.addResource(new File("hdfs-site.xml").toURI().toURL());
>
>>>>
>
>>>> conf.addResource(new File("core-site.xml").toURI().toURL());
>
>>>>
>
>>>> conf.addResource(new File("yarn-site.xml").toURI().toURL());
>
>>>>
>
>>>> conf.addResource(new File("hive-site.xml").toURI().toURL());
>
>>>>
>
>>>>
>
>>>>
>
>>>> conf.set("hadoop.security.authentication", "Kerberos");
>
>>>>
>
>>>> UserGroupInformation.setConfiguration(conf);
>
>>>>
>
>>>> UserGroupInformation.loginUserFromKeytab(principal, keytab);
>
>>>>
>
>>>>
>
>>>>
>
>>>> Class.forName("org.apache.hive.,jdbc.HiveDriver");
>
>>>>
>
>>>> Connection conn =DriverManager.getConnection(url);
>
>>>>
>
>>>>
>
>>>>
>
>>>> Statement stmt = conn.createStatement();
>
>>>>
>
>>>> String sql = "select * from testkerberos";
>
>>>>
>
>>>> ResultSet rs = stmt.executeQuery(sql);
>
>>>>
>
>>>> while (rs.next()) {
>
>>>>
>
>>>> system.out.println(rs.getString(1));
>
>>>>
>
>>>> }
>
>>>>
>
>>>>}
>
>>>>
>
>>>>
>
>>>>
>
>>>>Does anyone had the same problem? Or know how to solve it ?
>
>>>>
>
>>>>
>
>>>>
>
>>>>Thanks in advance.
>
>>>>
>
>>>>
>
>>>>
>
>>>>Maria.
>
>>>>
>
>>>>
>
>>>>
>
>
>
>
>
