This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 5b002c8 [SPARK-37293][TESTS] Remove explicit GC options from Scala
tests
5b002c8 is described below
commit 5b002c89d5b4683758769a500f53ea67404a36e8
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Nov 11 18:49:49 2021 -0800
[SPARK-37293][TESTS] Remove explicit GC options from Scala tests
### What changes were proposed in this pull request?
This PR aims to remove the explicit GC options in Scala tests.
### Why are the changes needed?
At Apache Spark 3.0, SPARK-29282 introduced the explicit GC options in
Scala tests to see the consistent result between Java8/Java11. Now, we have
Java 17 additionally and SPARK-37120 added Java 11/17 GitHub Action jobs.
This PR aims to use the default GC options for each JVM. This is a partial
revert of SPARK-29282.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the GitHub Action CIs without memory issues.
Closes #34560 from dongjoon-hyun/SPARK-37293.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
project/SparkBuild.scala | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index c647f31..68878ec 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -1135,7 +1135,6 @@ object TestSettings {
(Test / javaOptions) ++=
System.getProperties.asScala.filter(_._1.startsWith("spark"))
.map { case (k,v) => s"-D$k=$v" }.toSeq,
(Test / javaOptions) += "-ea",
- // SPARK-29282 This is for consistency between JDK8 and JDK11.
(Test / javaOptions) ++= {
val metaspaceSize = sys.env.get("METASPACE_SIZE").getOrElse("1300m")
val extraTestJavaArgs = Array("-XX:+IgnoreUnrecognizedVMOptions",
@@ -1156,7 +1155,7 @@ object TestSettings {
// to mock `j.u.Random`,
"-add-exports=java.base/jdk.internal.util.random=ALL-UNNAMED"
// is added. Should remove it when `mockito` can mock `j.u.Random`
directly.
"--add-exports=java.base/jdk.internal.util.random=ALL-UNNAMED").mkString(" ")
- s"-Xmx4g -Xss4m -XX:MaxMetaspaceSize=$metaspaceSize -XX:+UseParallelGC
-XX:-UseDynamicNumberOfGCThreads -XX:ReservedCodeCacheSize=128m
$extraTestJavaArgs"
+ s"-Xmx4g -Xss4m -XX:MaxMetaspaceSize=$metaspaceSize
-XX:ReservedCodeCacheSize=128m $extraTestJavaArgs"
.split(" ").toSeq
},
javaOptions ++= {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
