[
https://issues.apache.org/jira/browse/BEAM-4847?focusedWorklogId=126861&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-126861
]
ASF GitHub Bot logged work on BEAM-4847:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jul/18 20:36
Start Date: 24/Jul/18 20:36
Worklog Time Spent: 10m
Work Description: tweise closed pull request #6046: [BEAM-4847] Fix java
OOM errors during compile by setting maxWorkers, Xms and Xmx to reasonable
values
URL: https://github.com/apache/beam/pull/6046
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.test-infra/jenkins/CommonJobProperties.groovy
b/.test-infra/jenkins/CommonJobProperties.groovy
index 0af79eb7d7a..4373e1a4728 100644
--- a/.test-infra/jenkins/CommonJobProperties.groovy
+++ b/.test-infra/jenkins/CommonJobProperties.groovy
@@ -173,7 +173,8 @@ class CommonJobProperties {
}
}
- static void setGradleSwitches(context, maxWorkers =
Runtime.getRuntime().availableProcessors()) {
+ // Default maxWorkers is 12 to avoid jvm oom as in [BEAM-4847].
+ static void setGradleSwitches(context, maxWorkers = 12) {
def defaultSwitches = [
// Gradle log verbosity enough to diagnose basic build issues
"--info",
@@ -188,16 +189,9 @@ class CommonJobProperties {
// Ensure that parallel workers don't exceed total available memory.
- // TODO(BEAM-4230): OperatingSystemMXBeam incorrectly reports total
memory; hard-code for now
- // Jenkins machines are GCE n1-highmem-16, with 104 GB of memory
- // def os =
(com.sun.management.OperatingSystemMXBean)java.lang.management.ManagementFactory.getOperatingSystemMXBean()
- // def totalMemoryMb = os.getTotalPhysicalMemorySize() / (1024*1024)
- def totalMemoryMb = 104 * 1024
- // Jenkins uses 2 executors to schedule concurrent jobs, so ensure that
each executor uses only half the
- // machine memory.
- def totalExecutorMemoryMb = totalMemoryMb / 2
- def perWorkerMemoryMb = totalExecutorMemoryMb / maxWorkers
- context.switches("-Dorg.gradle.jvmargs=-Xmx${(int)perWorkerMemoryMb}m")
+ // For [BEAM-4847], hardcode Xms and Xmx to reasonable values (3g/8g).
+ context.switches("-Dorg.gradle.jvmargs=-Xms3g")
+ context.switches("-Dorg.gradle.jvmargs=-Xmx8g")
}
// Sets common config for PreCommit jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 126861)
Time Spent: 1.5h (was: 1h 20m)
> Java PreCommit is flaky: Expiring Daemon because JVM Tenured space is
> exhausted
> -------------------------------------------------------------------------------
>
> Key: BEAM-4847
> URL: https://issues.apache.org/jira/browse/BEAM-4847
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Reporter: Boyuan Zhang
> Assignee: Alan Myrvold
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Java PreCommit seems like flaky:
> https://builds.apache.org/job/beam_PreCommit_Java_Commit/505/console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
