[
https://issues.apache.org/jira/browse/HADOOP-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HADOOP-10274.
----------------------------
Resolution: Fixed
Fix Version/s: 2.3.0
3.0.0
Hadoop Flags: Reviewed
Committed to trunk, branch-2, and branch-2.3. Thanks for the patch Takeshi
Miao.
(I am not administrator in these parts so can't add takeshi.miao as a
contributor to assign him his issue.)
> Lower the logging level from ERROR to WARN for UGI.doAs method
> --------------------------------------------------------------
>
> Key: HADOOP-10274
> URL: https://issues.apache.org/jira/browse/HADOOP-10274
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
> Affects Versions: 1.0.4
> Environment: hadoop-1.0.4, hbase-0.94.16,
> krb5-server-1.6.1-31.el5_3.3, CentOS release 5.3 (Final)
> Reporter: takeshi.miao
> Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HADOOP-10274-trunk-v01.patch
>
>
> Recently we got the error msg "Request is a replay (34) - PROCESS_TGS" while
> we are using the HBase client API to put data into HBase-0.94.16 with
> krb5-1.6.1 enabled. The related msg as follows...
> {code}
> [2014-01-15
> 09:40:38,452][hbase-tablepool-1-thread-3][ERROR][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)):
> PriviledgedActionException as:takeshi_miao@LAB
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Request is a
> replay (34) - PROCESS_TGS)]
> [2014-01-15
> 09:40:38,453][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.logPriviledgedAction(UserGroupInformation.java:1143)):
> PriviledgedAction as:takeshi_miao@LAB
> from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> [2014-01-15
> 09:40:38,453][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.ipc.SecureClient](org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$1.run(SecureClient.java:213)):
> Exception encountered while connecting to the server :
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Request is a
> replay (34) - PROCESS_TGS)]
> [2014-01-15 09:40:38,454][hbase-tablepool-1-thread-3][INFO
> ][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.reloginFromTicketCache(UserGroupInformation.java:657)):
> Initiating logout for takeshi_miao@LAB
> [2014-01-15
> 09:40:38,454][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.logout(UserGroupInformation.java:154)):
> hadoop logout
> [2014-01-15 09:40:38,454][hbase-tablepool-1-thread-3][INFO
> ][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.reloginFromTicketCache(UserGroupInformation.java:667)):
> Initiating re-login for takeshi_miao@LAB
> [2014-01-15
> 09:40:38,455][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.login(UserGroupInformation.java:146)):
> hadoop login
> [2014-01-15
> 09:40:38,456][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:95)):
> hadoop login commit
> [2014-01-15
> 09:40:38,456][hbase-tablepool-1-thread-3][DEBUG][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:100)):
> using existing subject:[takeshi_miao@LAB, UnixPrincipal: takeshi_miao,
> UnixNumericUserPrincipal: 501, UnixNumericGroupPrincipal [Primary Group]:
> 501, UnixNumericGroupPrincipal [Supplementary Group]: 502, takeshi_miao@LAB]
> {code}
> Finally, we found that the HBase would doing the retry (5 * 10 times) and
> recovery this _'request is a replay (34)'_ issue, but based on the HBase user
> viewpoint, the error msg at first line may be frightful, as we were afraid
> there was any data loss occurring at the first sight...
> {code}
> [2014-01-15
> 09:40:38,452][hbase-tablepool-1-thread-3][ERROR][org.apache.hadoop.security.UserGroupInformation](org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)):
> PriviledgedActionException as:takeshi_miao@LAB
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Request is a
> replay (34) - PROCESS_TGS)]
> {code}
> So I'd like to suggest to change the logging level from '_ERROR_' to '_WARN_'
> for
> _o.a.hadoop.security.UserGroupInformation#doAs(PrivilegedExceptionAction<T>)_
> method
> {code}
> public <T> T doAs(PrivilegedExceptionAction<T> action
> ) throws IOException, InterruptedException {
> try {
> // ...
> } catch (PrivilegedActionException pae) {
> Throwable cause = pae.getCause();
> LOG.error("PriviledgedActionException as:"+this+" cause:"+cause); // I
> mean here
> // ...
> }
> }
> {code}
> Due to this method already throws _checked exception_s which can be handled
> by the client code, so the error may not really be an error if client code
> can handle it...such as this case.
> For more details pls refer to HBASE-10379
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
