takeshi.miao created HBASE-10379:
------------------------------------
Summary: Turn the msg "Request is a replay (34) - PROCESS_TGS"
from logging level ERROR to WARN
Key: HBASE-10379
URL: https://issues.apache.org/jira/browse/HBASE-10379
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.16
Reporter: takeshi.miao
Assignee: takeshi.miao
Priority: Minor
Hi All,
Recently we got the error msg "Request is a replay (34) - PROCESS_TGS" while we
are using the HBase client API to put data into HBase-0.94.16 with krb5-1.6.1
enabled. The related msg as follows...
{code}
[2014-01-15
09:40:38,452][hbase-tablepool-1-thread-3][ERROR][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)):
PriviledgedActionException as:takeshi_miao@LAB
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Request is a
replay (34) - PROCESS_TGS)]
[2014-01-15
09:40:38,453][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.logPriviledgedAction(UserGroupInformation.java:1143)):
PriviledgedAction as:takeshi_miao@LAB
from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2014-01-15
09:40:38,453][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.ipc.SecureClient](org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$1.run(SecureClient.java:213)):
Exception encountered while connecting to the server :
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException:
No valid credentials provided (Mechanism level: Request is a replay (34) -
PROCESS_TGS)]
[2014-01-15 09:40:38,454][hbase-tablepool-1-thread-3][INFO
][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.reloginFromTicketCache(UserGroupInformation.java:657)):
Initiating logout for takeshi_miao@LAB
[2014-01-15
09:40:38,454][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.logout(UserGroupInformation.java:154)):
hadoop logout
[2014-01-15 09:40:38,454][hbase-tablepool-1-thread-3][INFO
][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.reloginFromTicketCache(UserGroupInformation.java:667)):
Initiating re-login for takeshi_miao@LAB
[2014-01-15
09:40:38,455][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.login(UserGroupInformation.java:146)):
hadoop login
[2014-01-15
09:40:38,456][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:95)):
hadoop login commit
[2014-01-15
09:40:38,456][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:100)):
using existing subject:[takeshi_miao@LAB, UnixPrincipal: takeshi_miao,
UnixNumericUserPrincipal: 501, UnixNumericGroupPrincipal [Primary Group]: 501,
UnixNumericGroupPrincipal [Supplementary Group]: 502, takeshi_miao@LAB]
{code}
At the beginning, we were worry about the "data loss" occurring while we found
the "Request is a replay (34) - PROCESS_TGS" (especially it is the ERROR level)
in log, but after code study, this is basically *NOT* a data loss issue due to
HBase client API would try 5 times internally (o.a.h.hbase.ipc.SecureClient,
L#296, a one thread) and also 10 times of retry externally
(o.a.h.hbase.client.HConnectionManager, L#1661, for all failed thread), The
HTable API would also throw IOEcxeption to client code if any thread still fail
after these retries.
Based on HBase users' viewpoint as us, we think this is better to change the
logging level from 'ERROR' to 'WARN', due to the 'ERROR' level had been
confused us for a while...But this code change may need to change boht HBase
code and Hadoop code as well; so I am wondering how community think about this
small thing but may be important to the pure HBase users.
mailing list
http://mail-archives.apache.org/mod_mbox/hbase-user/201401.mbox/%3CCADcMMgGiEyho0HGwgbfOUS78ymDpCo5Q0PStWAPUk40W%3DPfcFQ%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
