[
https://issues.apache.org/jira/browse/BEAM-4847?focusedWorklogId=126874&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-126874
]
ASF GitHub Bot logged work on BEAM-4847:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jul/18 20:52
Start Date: 24/Jul/18 20:52
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #6046:
[BEAM-4847] Fix java OOM errors during compile by setting maxWorkers, Xms and
Xmx to reasonable values
URL: https://github.com/apache/beam/pull/6046#discussion_r204907065
##########
File path: .test-infra/jenkins/CommonJobProperties.groovy
##########
@@ -188,16 +189,9 @@ class CommonJobProperties {
// Ensure that parallel workers don't exceed total available memory.
- // TODO(BEAM-4230): OperatingSystemMXBeam incorrectly reports total
memory; hard-code for now
- // Jenkins machines are GCE n1-highmem-16, with 104 GB of memory
- // def os =
(com.sun.management.OperatingSystemMXBean)java.lang.management.ManagementFactory.getOperatingSystemMXBean()
- // def totalMemoryMb = os.getTotalPhysicalMemorySize() / (1024*1024)
- def totalMemoryMb = 104 * 1024
- // Jenkins uses 2 executors to schedule concurrent jobs, so ensure that
each executor uses only half the
- // machine memory.
- def totalExecutorMemoryMb = totalMemoryMb / 2
- def perWorkerMemoryMb = totalExecutorMemoryMb / maxWorkers
- context.switches("-Dorg.gradle.jvmargs=-Xmx${(int)perWorkerMemoryMb}m")
+ // For [BEAM-4847], hardcode Xms and Xmx to reasonable values (3g/8g).
Review comment:
Won't we blow out the memory usage if we have two executors per machine.
24 workers * 8gbs > 104 gbs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 126874)
Time Spent: 1h 40m (was: 1.5h)
> Java PreCommit is flaky: Expiring Daemon because JVM Tenured space is
> exhausted
> -------------------------------------------------------------------------------
>
> Key: BEAM-4847
> URL: https://issues.apache.org/jira/browse/BEAM-4847
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Reporter: Boyuan Zhang
> Assignee: Alan Myrvold
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Java PreCommit seems like flaky:
> https://builds.apache.org/job/beam_PreCommit_Java_Commit/505/console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
