MartijnVisser commented on PR #21128:
URL: https://github.com/apache/flink/pull/21128#issuecomment-1356078148
> The question is why Hadoop3 tests are failing there w/o any changes. This
can be challenging though 😅
So the tests pass on the CI for all the Hadoop 2 config. The problem on the
CI is that the upgrade to Hadoop 3 is causing the generic `HadoopUtilsTest` to
fail with:
```
java.lang.IllegalArgumentException: Can't get Kerberos realm
at
org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:71)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:319)
at
org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:365)
at
org.apache.flink.runtime.util.HadoopUtilsTest.testShouldReturnTrueIfTicketCacheIsNotUsed(HadoopUtilsTest.java:116)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at
com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: java.lang.IllegalArgumentException: Null realm name (601)
at
java.security.jgss/javax.security.auth.kerberos.KerberosPrincipal.<init>(KerberosPrincipal.java:179)
at
org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:108)
at
org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:69)
... 34 more
```
```
Dec 16 23:12:50 [ERROR] Errors:
Dec 16 23:12:50 [ERROR]
HadoopUtilsTest.testShouldReturnFalseWhenNoKerberosCredentialsOrDelegationTokens:58
» IllegalArgument
Dec 16 23:12:50 [ERROR]
HadoopUtilsTest.testShouldReturnTrueIfTicketCacheIsNotUsed:116 » IllegalArgument
Dec 16 23:12:50 [ERROR]
HadoopUtilsTest.testShouldReturnTrueWhenDelegationTokenIsPresent:75 »
IllegalArgument
Dec 16 23:12:50 [ERROR]
HadoopUtilsTest.testShouldReturnTrueWhenKerberosCredentialsArePresent:90 »
IllegalArgument
Dec 16 23:12:50 [INFO]
Dec 16 23:12:50 [ERROR] Tests run: 72, Failures: 0, Errors: 4, Skipped: 0
```
If we can figure out what's causing that and see if it can be fixed in a
generic way (for both the Hadoop 2 and Hadoop 3 profiles), we are getting
pretty damn close 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
