This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 85b593f21b75 [SPARK-56833][TESTS] Add `-XX:+EnableDynamicAgentLoading`
to test JVM options to suppress JEP 451 warnings
85b593f21b75 is described below
commit 85b593f21b7500135555aceb3679fa645b8f1e82
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue May 12 09:22:12 2026 -0700
[SPARK-56833][TESTS] Add `-XX:+EnableDynamicAgentLoading` to test JVM
options to suppress JEP 451 warnings
### What changes were proposed in this pull request?
Add `-XX:+EnableDynamicAgentLoading` to the test-only JVM options in both
SBT (`project/SparkBuild.scala`) and Maven (`pom.xml`).
### Why are the changes needed?
Per [JEP 451: Prepare to Disallow the Dynamic Loading of
Agents](https://openjdk.org/jeps/451), JDK 21+ warns when a Java agent is
loaded dynamically (e.g. Mockito self-attaching ByteBuddy), and a future
release will disallow it by default. Setting this flag on the test JVM silences
the warning today and keeps tests working once the default flips. Production
JVM options are intentionally untouched since Spark itself does not dynamically
attach agents.
Currently, Apache Spark CIs show the following message at ERROR level.
- Java 21:
https://github.com/apache/spark/actions/runs/25592460535/job/75172442863
- Java 25:
https://github.com/apache/spark/actions/runs/25664584995/job/75333859273
```
[info] Test run started (JUnit Jupiter)
[info] Test
org.apache.spark.network.TransportRequestHandlerSuite#handleMergedBlockMetaRequest()
started
[error] Mockito is currently self-attaching to enable the
inline-mock-maker. This will no longer work in future releases of the JDK.
Please add Mockito as an agent to your build as described in Mockito's
documentation:
https://javadoc.io/doc/org.mockito/mockito-core/latest/org.mockito/org/mockito/Mockito.html#0.3
WARNING: A Java agent has been loaded dynamically
(/home/runner/.cache/coursier/v1/https/maven-central.storage-download.googleapis.com/maven2/net/bytebuddy/byte-buddy-agent/1.18.4/byte-buddy-agent-1.18.4.jar)
WARNING: If a serviceability tool is in use, please run with
-XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with
-Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a
future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader
classes because bootstrap classpath has been appended
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs and manual tests.
**BEFORE**
```
$ build/sbt "network-common/testOnly *.TransportRequestHandlerSuite" >
/dev/null
Using SPARK_LOCAL_IP=localhost
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by
scala.runtime.LazyVals$
(file:/Users/dongjoon/APACHE/spark-merge/build/sbt-launch-1.12.8.jar)
WARNING: Please consider reporting this to the maintainers of class
scala.runtime.LazyVals$
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future
release
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: package sun.security.action not in java.base
WARNING: A Java agent has been loaded dynamically
(/Users/dongjoon/.m2/repository/net/bytebuddy/byte-buddy-agent/1.18.4/byte-buddy-agent-1.18.4.jar)
WARNING: If a serviceability tool is in use, please run with
-XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with
-Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a
future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader
classes because bootstrap classpath has been appended
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::allocateMemory has been called by
io.netty.util.internal.PlatformDependent0$2
(file:/Users/dongjoon/.m2/repository/io/netty/netty-common/4.2.12.Final/netty-common-4.2.12.Final.jar)
WARNING: Please consider reporting this to the maintainers of class
io.netty.util.internal.PlatformDependent0$2
WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
```
**AFTER**
```
$ build/sbt "network-common/testOnly *.TransportRequestHandlerSuite" >
/dev/null
Using SPARK_LOCAL_IP=localhost
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by
scala.runtime.LazyVals$
(file:/Users/dongjoon/APACHE/spark-merge/.claude/worktrees/youthful-sanderson-0fc91a/build/sbt-launch-1.12.8.jar)
WARNING: Please consider reporting this to the maintainers of class
scala.runtime.LazyVals$
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future
release
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: package sun.security.action not in java.base
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader
classes because bootstrap classpath has been appended
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::allocateMemory has been called by
io.netty.util.internal.PlatformDependent0$2
(file:/Users/dongjoon/.m2/repository/io/netty/netty-common/4.2.12.Final/netty-common-4.2.12.Final.jar)
WARNING: Please consider reporting this to the maintainers of class
io.netty.util.internal.PlatformDependent0$2
WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.7 (1M context)
Closes #55825 from dongjoon-hyun/SPARK-56833.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 08a29819bf4093fb756f3160b49e5f99500323a5)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
pom.xml | 1 +
project/SparkBuild.scala | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 8e8517106804..57faf6d78199 100644
--- a/pom.xml
+++ b/pom.xml
@@ -342,6 +342,7 @@
-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE
-Dio.netty.noUnsafe=false
--enable-native-access=ALL-UNNAMED
+ -XX:+EnableDynamicAgentLoading
</extraJavaTestArgs>
<mariadb.java.client.version>3.5.7</mariadb.java.client.version>
<mysql.connector.version>9.6.0</mysql.connector.version>
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 66947a58ac59..49556aa1df18 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -1950,7 +1950,8 @@ object TestSettings {
"-Dio.netty.allocator.type=pooled",
"-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE",
"-Dio.netty.noUnsafe=false",
- "--enable-native-access=ALL-UNNAMED").mkString(" ")
+ "--enable-native-access=ALL-UNNAMED",
+ "-XX:+EnableDynamicAgentLoading").mkString(" ")
s"-Xmx$heapSize -Xss4m -XX:MaxMetaspaceSize=$metaspaceSize
-XX:ReservedCodeCacheSize=128m -Dfile.encoding=UTF-8 $extraTestJavaArgs"
.split(" ").toSeq
},
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
