[
https://issues.apache.org/jira/browse/HADOOP-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720086#comment-17720086
]
Vinay Devadiga commented on HADOOP-18716:
-----------------------------------------
Hi [~ayushtkn] i dont have band width to raise PR , but here are most of the
fixes explanation here are there tested it out . Any body can raise PR .
h3. Appendix
#1.The first major error which we encountered was the following stack trace -
java.lang.NoClassDefFoundError: Could not initialize class
com.google.inject.internal.cglib.core.$ReflectUtils
at
com.google.inject.internal.cglib.reflect.$FastClassEmitter.<init>(FastClassEmitter.java:67)
at
com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:72)
This error seems to be the lack of support of the current guice version 4.0 for
jre 17 . so after upgrading the guice version to 5.1.0 reduced the {*}errors by
the 300{*}.
*Ref:*
[https://github.com/google/guice/issues/1536]
[https://github.com/google/guice/pull/1529/files]
--<guice.version>4.0</guice.version>
++<guice.version>5.1.0</guice.version>
------------------------------------------------------------------------------------------------------
2 . The second major error which we encountered the child jvms spawned by the
maven do not have major add-opens which were causing various failures .
```
1. java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesAttempts,TestJerseyClient
2. java.io.IOException: 0.0.0.0:43529: Unexpected HTTP response:
code=500 != 200, op=MKDIRS,
message=Server Error
4. org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
6. java.lang.Exception: Unexpected exception,
expected<org.apache.hadoop.yarn.webapp.WebAppException> but
was<java.lang.NoClassDefFoundError>
7. org.apache.hadoop.service.ServiceStateException:
org.apache.hadoop.yarn.exceptions.YarnException: Failed to
initialize queues
8. org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.lang.ExceptionInInitializerError
```
and many more
To fix this we use the arg line child property of
<maven-surefire-plugin.argLine>-Xmx2048m
-XX:+HeapDumpOnOutOfMemoryError</maven-surefire-plugin.argLine>
We add the following - add-opens in the arg line and in the child projects
--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.math=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED
--add-opens=java.base/java.text=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
This fix brought down the errors by 2000. So the current count is {*}reduced to
500{*}.
-------------------------------------------------------------------------------------------------------
3. There were various mockito exceptions of these following stack trace
java.lang.NullPointerException: Cannot enter synchronized block because
"this.lock" is null
Mockito cannot mock this class: class com.sun.jersey.api.client.ClientResponse.
If you're not sure why you're getting this error, please report to the mailing
list.
Java : 17
JVM vendor name : Amazon.com Inc.
JVM vendor version : 17.0.6+10-LTS
JVM name : OpenJDK 64-Bit Server VM
JVM version : 17.0.6+10-LTS
JVM info : mixed mode, sharing
OS name : Linux
OS version : 4.14.309-231.529.amzn2.x86_64
You are seeing this disclaimer because Mockito is configured to create inlined
mocks.
You can learn about inline mocks and their limitations under item #39 of the
Mockito class javadoc.
Underlying exception : org.mockito.exceptions.base.MockitoException: Could not
modify all classes [class com.sun.jersey.api.client.ClientResponse, class
java.lang.Object]
etc .These errors got fix after updating the mockito-core version and its scope
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.28.2</version>
<version>3.11.2</version>
<scope>provided</scope>
Other mockito exceptions are failing with following stack trace -
Caused by: java.lang.UnsatisfiedLinkError:
jdk.internal.misc.Unsafe.registerNatives()V
at jdk.internal.misc.Unsafe.registerNatives(Native Method)
at jdk.internal.misc.Unsafe.<clinit>(Unsafe.java:57)
at java.base/java.lang.Class.forName0(Native Method)
This is surely because we have not access to {{Unsafe}} .
Ref: [https://github.com/jboss-javassist/javassist/issues/227]
[https://github.com/mockito/mockito/issues/1207]
[.
https://github.com/mockito/mockito/issues/2164|https://github.com/mockito/mockito/issues/2164]
This brought down *error failures to 200* .
# There were some other minor errors like for [MetricsInvariantChecker
tests|https://code.amazon.com/packages/Aws157HadoopGit/blobs/4b4b79f64a65146acdc5586428e5438aa1639867/--/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/invariants/MetricsInvariantChecker.java]
etc which used javascript engine for their testing, which is not supported in
JRE 17 so the fix was add the following dependency and use graal.js as an
alternative.
Ref:
[https://stackoverflow.com/questions/71481562/use-javascript-scripting-engine-in-java-17]
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js</artifactId>
<version>22.0.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js-scriptengine</artifactId>
<version>22.0.0</version>
<scope>provided</scope>
</dependency> --this.scriptEngine = (Compilable)
manager.getEngineByName("JavaScript");
++this.scriptEngine = (Compilable) manager.getEngineByName("graal.js");
There were also some other failures -
cannot access class sun.nio.ch.DirectBuffer (in module java.base) because
module java.base does not export sun.nio.ch to unnamed module @0x59d016c9
This caused fixed by adding -
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
--add-opens=java.base/sun.nio.cs=ALL-UNNAMED \
Ref:
[https://stackoverflow.com/questions/73465937/apache-spark-3-3-0-breaks-on-java-17-with-cannot-access-class-sun-nio-ch-direct]
---------------------------------------------------------------------------------------------------
5.The next set of errors which we fixed were IPC failures for these following
classes TestIPC,TestAsyncIPC and TestRPCServerShutdown .
We encountered the following stack trace -
Caused by: java.lang.NoClassDefFoundError:
org/bouncycastle/jce/provider/BouncyCastleProvider
at
io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:244)
... 98 more
Caused by: java.lang.ClassNotFoundException:
org.bouncycastle.jce.provider.BouncyCastleProvider
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:606)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:168)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
... 99 more
This was fixed by adding the following dependency which was missing JRE 17
runtime for Netty -{*}{{org.bouncycastle:bcpkix-jdk15on:1.65}}{*}
Ref: [https://github.com/netty/netty/issues/10317]
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<scope>test</scope>
</dependency>
This reduced the {*}error count to 96{*}.
---------------------------------------------------------------------------------------------------------
These are very rough numbers but this solves most of the issue .
For some errors : fix is this have not tried it
https://github.com/apache/hbase/pull/3443/files
----------------------------------------------------
Just trying to contribute to the community and my findings .
> [JDK-17] Failed unit tests , with Java 17 runtime and compiled Java 8
> ---------------------------------------------------------------------
>
> Key: HADOOP-18716
> URL: https://issues.apache.org/jira/browse/HADOOP-18716
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay Devadiga
> Priority: Critical
>
> Compiled Hadoop - Hadoop branch 3.3.3
> mvn clean install - DskipTests=True
> Java_Home -> points to Java-8
> maven version - 3.8.8 (Quite latest)
>
> Ran various whole test suit on my private cloud environment -
> Changed Java_Home to Java-17
>
> mvn surefire:test
>
> Out of 22k tests - 2.5 k tests failed .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
