This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f4bb6528596b [SPARK-47115][INFRA] Use larger memory for Maven builds
f4bb6528596b is described below
commit f4bb6528596b6809391664e594be8c7a154529d8
Author: Hyukjin Kwon <gurwls...@apache.org>
AuthorDate: Wed Feb 21 16:06:26 2024 +0900
[SPARK-47115][INFRA] Use larger memory for Maven builds
### What changes were proposed in this pull request?
This PR proposes to use bigger memory during Maven builds. GitHub Actions
runners now have more memory than before
(https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners)
so we can increase.
https://github.com/HyukjinKwon/spark/actions/runs/7984135094/job/21800463337
### Why are the changes needed?
For stable Maven builds.
Some tests consistently fail:
```
*** RUN ABORTED ***
An exception or error caused a run to abort: unable to create native
thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out
of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:1553)
at java.base/java.lang.System$2.start(System.java:2577)
at
java.base/jdk.internal.vm.SharedThreadContainer.start(SharedThreadContainer.java:152)
at
java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:953)
at
java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
at
org.apache.spark.rpc.netty.SharedMessageLoop.$anonfun$threadpool$1(MessageLoop.scala:128)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
at
org.apache.spark.rpc.netty.SharedMessageLoop.<init>(MessageLoop.scala:127)
at
org.apache.spark.rpc.netty.Dispatcher.sharedLoop$lzycompute(Dispatcher.scala:46)
...
Warning: The requested profile "volcano" could not be activated because it
does not exist.
Warning: The requested profile "hive" could not be activated because it
does not exist.
Error: Failed to execute goal
org.scalatest:scalatest-maven-plugin:2.2.0:test (test) on project
spark-core_2.13: There are test failures -> [Help 1]
Error:
Error: To see the full stack trace of the errors, re-run Maven with the -e
switch.
Error: Re-run Maven using the -X switch to enable full debug logging.
Error:
Error: For more information about the errors and possible solutions,
please read the following articles:
Error: [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Error:
Error: After correcting the problems, you can resume the build with the
command
Error: mvn <args> -rf :spark-core_2.13
Error: Process completed with exit code 1.
```
### Does this PR introduce _any_ user-facing change?
No, dev-only
### How was this patch tested?
Will monitor the scheduled jobs. It's a simple memory configuration change.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45195 from HyukjinKwon/bigger-macos.
Authored-by: Hyukjin Kwon <gurwls...@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
.github/workflows/maven_test.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/maven_test.yml b/.github/workflows/maven_test.yml
index d63066a521f9..b2f0118d2e7b 100644
--- a/.github/workflows/maven_test.yml
+++ b/.github/workflows/maven_test.yml
@@ -185,7 +185,7 @@ jobs:
- name: Run tests
env: ${{ fromJSON(inputs.envs) }}
run: |
- export MAVEN_OPTS="-Xss64m -Xmx4g -Xms4g
-XX:ReservedCodeCacheSize=128m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
+ export MAVEN_OPTS="-Xss64m -Xmx6g -Xms6g
-XX:ReservedCodeCacheSize=128m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
export MAVEN_CLI_OPTS="--no-transfer-progress"
export JAVA_VERSION=${{ matrix.java }}
# Replace with the real module name, for example,
connector#kafka-0-10 -> connector/kafka-0-10
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
