This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e696c5ebc6a4 [SPARK-54830][SQL][TESTS][FOLLOWUP] Increase `-Xmx` for
the sql module to 6g for maven test
e696c5ebc6a4 is described below
commit e696c5ebc6a41b66d2f7047dae7059d64370067d
Author: yangjie01 <[email protected]>
AuthorDate: Mon Jan 19 17:17:51 2026 +0800
[SPARK-54830][SQL][TESTS][FOLLOWUP] Increase `-Xmx` for the sql module to
6g for maven test
### What changes were proposed in this pull request?
When testing the SQL module using sbt, SPARK-54830 configure `-Xmx` to 6g
to prevent OOM.
https://github.com/apache/spark/blob/c6ca25a8028bae0b7b8f6a1bf0105b1dd236ceed/project/SparkBuild.scala#L1325-L1327
This pr synchronizes this modification to the Maven side to avoid OOM
occurrences when testing the SQL module using Maven.
### Why are the changes needed?
Avoid OOM that occur during Maven daily tests:
- https://github.com/apache/spark/actions/runs/21112257952/job/60712578744
```
14:14:21.568 ERROR org.apache.spark.executor.Executor: Exception in task
1.0 in stage 1710.0 (TID 7739)
java.lang.OutOfMemoryError: Java heap space
at
org.apache.spark.shuffle.checksum.ShuffleChecksumSupport.getChecksumValues(ShuffleChecksumSupport.java:39)
at
org.apache.spark.util.collection.ExternalSorter.getChecksumValues(ExternalSorter.scala:95)
at
org.apache.spark.util.collection.ExternalSorter.getChecksums(ExternalSorter.scala:149)
at
org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:86)
at
org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:57)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:111)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
14:14:24.800 ERROR org.apache.spark.util.SparkUncaughtExceptionHandler:
Uncaught exception in thread Thread[Executor task launch worker for task 1.0 in
stage 1710.0 (TID 7739),5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.spark.shuffle.checksum.ShuffleChecksumSupport.getChecksumValues(ShuffleChecksumSupport.java:39)
at
org.apache.spark.util.collection.ExternalSorter.getChecksumValues(ExternalSorter.scala:95)
at
org.apache.spark.util.collection.ExternalSorter.getChecksums(ExternalSorter.scala:149)
at
org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:86)
at
org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:57)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:111)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
14:14:24.800 WARN org.apache.spark.scheduler.TaskSetManager: Lost task 1.0
in stage 1710.0 (TID 7739) (localhost executor driver):
java.lang.OutOfMemoryError: Java heap space
at
org.apache.spark.shuffle.checksum.ShuffleChecksumSupport.getChecksumValues(ShuffleChecksumSupport.java:39)
at
org.apache.spark.util.collection.ExternalSorter.getChecksumValues(ExternalSorter.scala:95)
at
org.apache.spark.util.collection.ExternalSorter.getChecksums(ExternalSorter.scala:149)
at
org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:86)
at org.apache....
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass Github Actions
- check maven tests with Github Actions:
https://github.com/LuciferYang/spark/actions/runs/21125802102/job/60746762507
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53847 from LuciferYang/check-maven-test.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
sql/core/pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sql/core/pom.xml b/sql/core/pom.xml
index ab6a8f8182e6..c5201065c604 100644
--- a/sql/core/pom.xml
+++ b/sql/core/pom.xml
@@ -344,7 +344,7 @@
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<configuration>
- <argLine>-ea -Xmx4g -Xss4m
-XX:ReservedCodeCacheSize=${CodeCacheSize} ${extraJavaTestArgs}
-Dio.netty.tryReflectionSetAccessible=true</argLine>
+ <argLine>-ea -Xmx6g -Xss4m
-XX:ReservedCodeCacheSize=${CodeCacheSize} ${extraJavaTestArgs}
-Dio.netty.tryReflectionSetAccessible=true</argLine>
</configuration>
</plugin>
<plugin>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
