[
https://issues.apache.org/jira/browse/HDFS-14461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896267#comment-16896267
]
Eric Yang commented on HDFS-14461:
----------------------------------
[~hexiaoqiao] The test cases fail on my system same as Jenkins reported.
Please make sure that your .m2 maven cache is cleared to ensure your test
results are accurate.
TestRouterWithSecureStartup#testStartupWithoutSpnegoPrincipal tests for invalid
Spnego principal setup by unsetting dfs.web.authentication.kerberos.keytab
configuration. The test case can be updated to looking for
hadoop.http.authentication.kerberos.principal because SecurityConfUtil has been
updated to use the globally consistent configuration for referencing Spnego
keytab setup.
TestRouterFaultTolerant#testWriteWithFailedSubcluster also failed because the
test case is written for simple security. SecurityConfUtil will turn on SPNEGO
authentication to http protocol when this patch is applied. This caused client
to unable to talk to namenode to get block locations if client does not send
Authentication negotiation header.
> RBF: Fix intermittently failing kerberos related unit test
> ----------------------------------------------------------
>
> Key: HDFS-14461
> URL: https://issues.apache.org/jira/browse/HDFS-14461
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: CR Hota
> Assignee: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-14461.001.patch, HDFS-14461.002.patch
>
>
> TestRouterHttpDelegationToken#testGetDelegationToken fails intermittently. It
> may be due to some race condition before using the keytab that's created for
> testing.
>
> {code:java}
> Failed
> org.apache.hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken.testGetDelegationToken
> Failing for the past 1 build (Since
> [!https://builds.apache.org/static/1e9ab9cc/images/16x16/red.png!
> #26721|https://builds.apache.org/job/PreCommit-HDFS-Build/26721/] )
> [Took 89
> ms.|https://builds.apache.org/job/PreCommit-HDFS-Build/26721/testReport/org.apache.hadoop.hdfs.server.federation.security/TestRouterHttpDelegationToken/testGetDelegationToken/history]
>
> Error Message
> org.apache.hadoop.security.KerberosAuthException: failure to login: for
> principal: router/[email protected] from keytab
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/target/test/data/SecurityConfUtil/test.keytab
> javax.security.auth.login.LoginException: Integrity check on decrypted field
> failed (31) - PREAUTH_FAILED
> h3. Stacktrace
> org.apache.hadoop.service.ServiceStateException:
> org.apache.hadoop.security.KerberosAuthException: failure to login: for
> principal: router/[email protected] from keytab
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/target/test/data/SecurityConfUtil/test.keytab
> javax.security.auth.login.LoginException: Integrity check on decrypted field
> failed (31) - PREAUTH_FAILED at
> org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
> at
> org.apache.hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken.setup(TestRouterHttpDelegationToken.java:99)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: org.apache.hadoop.security.KerberosAuthException: failure to
> login: for principal: router/[email protected] from keytab
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/target/test/data/SecurityConfUtil/test.keytab
> javax.security.auth.login.LoginException: Integrity check on decrypted field
> failed (31) - PREAUTH_FAILED at
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2008)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1376)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1156)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315) at
> org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:159)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> ... 27 more Caused by: javax.security.auth.login.LoginException: Integrity
> check on decrypted field failed (31) - PREAUTH_FAILED at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:804)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at
> javax.security.auth.login.LoginContext.login(LoginContext.java:587) at
> org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2087)
> at
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1998)
> ... 32 more Caused by: KrbException: Integrity check on decrypted field
> failed (31) - PREAUTH_FAILED at
> sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:82) at
> sun.security.krb5.KrbAsReqBuilder.send(KrbAsReqBuilder.java:316) at
> sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:361) at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:776)
> ... 46 more Caused by: KrbException: Identifier doesn't match expected value
> (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at
> sun.security.krb5.internal.ASRep.init(ASRep.java:64) at
> sun.security.krb5.internal.ASRep.<init>(ASRep.java:59) at
> sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:60) ... 49 more
> h3. Standard Output
> 2019-04-28 05:53:23,957 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(225)) - Configuration: 2019-04-28 05:53:23,957 [Listener
> at localhost/39018] INFO minikdc.MiniKdc (MiniKdc.java:<init>(226)) -
> --------------------------------------------------------------- 2019-04-28
> 05:53:23,957 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(228)) - debug: false 2019-04-28 05:53:23,957 [Listener
> at localhost/39018] INFO minikdc.MiniKdc (MiniKdc.java:<init>(228)) -
> transport: TCP 2019-04-28 05:53:23,957 [Listener at localhost/39018] INFO
> minikdc.MiniKdc (MiniKdc.java:<init>(228)) - max.ticket.lifetime: 86400000
> 2019-04-28 05:53:23,958 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(228)) - org.name: EXAMPLE 2019-04-28 05:53:23,958
> [Listener at localhost/39018] INFO minikdc.MiniKdc (MiniKdc.java:<init>(228))
> - kdc.port: 0 2019-04-28 05:53:23,958 [Listener at localhost/39018] INFO
> minikdc.MiniKdc (MiniKdc.java:<init>(228)) - org.domain: COM 2019-04-28
> 05:53:23,958 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(228)) - max.renewable.lifetime: 604800000 2019-04-28
> 05:53:23,958 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(228)) - instance: DefaultKrbServer 2019-04-28
> 05:53:23,958 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(228)) - kdc.bind.address: localhost 2019-04-28
> 05:53:23,959 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:<init>(230)) -
> --------------------------------------------------------------- 2019-04-28
> 05:53:23,961 [Listener at localhost/39018] INFO minikdc.MiniKdc
> (MiniKdc.java:start(285)) - MiniKdc started. 2019-04-28 05:53:24,014
> [pool-10-thread-1] INFO request.KdcRequest (KdcRequest.java:preauth(651)) -
> The preauth data is empty. 2019-04-28 05:53:24,015 [pool-10-thread-1] INFO
> server.KdcHandler (KdcHandler.java:handleRecoverableException(177)) - KRB
> error occurred while processing request:Additional pre-authentication
> required 2019-04-28 05:53:24,025 [Listener at localhost/39018] INFO
> service.AbstractService (AbstractService.java:noteFailure(267)) - Service
> org.apache.hadoop.hdfs.server.federation.router.Router failed in state INITED
> org.apache.hadoop.security.KerberosAuthException: failure to login: for
> principal: router/[email protected] from keytab
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/target/test/data/SecurityConfUtil/test.keytab
> javax.security.auth.login.LoginException: Integrity check on decrypted field
> failed (31) - PREAUTH_FAILED at
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2008)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1376)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1156)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315) at
> org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:159)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken.setup(TestRouterHttpDelegationToken.java:99)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: javax.security.auth.login.LoginException: Integrity check on
> decrypted field failed (31) - PREAUTH_FAILED at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:804)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at
> javax.security.auth.login.LoginContext.login(LoginContext.java:587) at
> org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2087)
> at
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1998)
> ... 32 more Caused by: KrbException: Integrity check on decrypted field
> failed (31) - PREAUTH_FAILED at
> sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:82) at
> sun.security.krb5.KrbAsReqBuilder.send(KrbAsReqBuilder.java:316) at
> sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:361) at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:776)
> ... 46 more Caused by: KrbException: Identifier doesn't match expected value
> (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at
> sun.security.krb5.internal.ASRep.init(ASRep.java:64) at
> sun.security.krb5.internal.ASRep.<init>(ASRep.java:59) at
> sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:60) ... 49 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
