> > Anyway long story short: yes, our build could be faster with better > hardware. Just not very clear what that hardware should be.
We don't know what is running under the hood in https://ci-builds.apache.org/ for on-demand workers, but I do see other projects that have specific controller types: https://jenkins-ccos.apache.org/ >From the numbers so far, it seems that anything bigger than AWS t2.xlarge (4 vCpus, 16 GB ram) can speed by half the current timing executor in the apache infra is taking https://builds.apache.org/job/TomEE/job/master-build-full/buildTimeTrend Based on Jonathan's feedback, I reviewed, and the current https://builds.apache.org/job/TomEE/job/master-build-full is using in MAVEN_OPTS = -Xmx1536m I wonder if the result shared in this thread have specific MAVEN_OPTS values, mine are = -Xms2g -Xmx8g I updated https://builds.apache.org/job/TomEE/job/builder-info/ to show the worker memory, and it seems we have 18 GB free out of 94 GB. [1], but this is distributed amount of other projects running in that worker, so there is no guarantee we will always have X or Y GB of RAM available. I don't have access to check the runner's limits peer node in the entire builds.apache.org. and https://cwiki.apache.org/confluence/display/INFRA/Jenkins+node+labels documentation seems to be outdated. Today's build already finished https://builds.apache.org/job/TomEE/job/master-build-full/774/ and took ~ 4 hour. I updated https://builds.apache.org/job/TomEE/job/master-build-full/ with the following changes: a) Changed MAVEN_OPTS-Xmx1536m to MAVEN_OPTS=-Xmx4096m b) Changed -U --show-version --fail-at-end clean install -Pall-adapters -Dfile.encoding=UTF-8 to -U help:system --show-version --fail-at-end clean install -Pall-adapters -Dfile.encoding=UTF-8 The only change is I added help:system maven argument to maven to double-check the OTPS are read. c) Build https://builds.apache.org/job/TomEE/job/master-build-full/776/ is currently running with a) and b) applied, I don't expect to see major improvements even if we are now using 4GB and the load on the shared server is unpredictable. After building https://builds.apache.org/job/TomEE/job/master-build-full/776/ I plan to revert to previous job configurations if need it. [1] free -g total used free shared buff/cache available Mem: 94 10 18 0 64 82 El vie, 14 oct 2022 a las 8:18, Jonathan Gallimore (< jonathan.gallim...@gmail.com>) escribió: > Taking the commands very literally, the timings for 2 machines here are > below. I'm curious though - are there any JAVA_OPTS or MAVEN_OPTS we should > use here (e.g. max heap)? The slower machine here seems to be running at an > absolute crawl 2/3rds of the way through the build here with the default > settings. The result below is after a reboot and MAVEN_OPTS="-Xmx4g". > > Laptop (WSL2/Ubuntu on Windows 11, i7-12700H, 32GB Ram): > > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 01:36 h > [INFO] Finished at: 2022-10-14T11:10:08+01:00 > [INFO] > ------------------------------------------------------------------------ > > Dell Optiplex 7040 "Server" machine (Ubuntu 20.04.5 LTS, i7-6700T, 16GB > Ram): > > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 01:57 h > [INFO] Finished at: 2022-10-14T15:16:58+01:00 > [INFO] > ------------------------------------------------------------------------ > > On Fri, Oct 14, 2022 at 12:13 AM David Blevins <david.blev...@gmail.com> > wrote: > > > Alright, here are some build times for the runs I've done. Results are > > quite varied. > > > > c1.xlarge : [INFO] Total time: 03:59 h > > t2.medium : [INFO] Total time: 05:05 h > > t2.xlarge : [INFO] Total time: 02:30 h > > t3.medium : [INFO] Total time: 03:21 h > > my.laptop : [INFO] Total time: 01:56 h > > > > Specs for the AWS instances are public. Here are the stats for my > laptop: > > > > MacBook Pro (16-inch, 2019) > > 2.4 GHz 8-Core Intel Core i9 > > Memory 64 GB 2667 MHz DDR4 > > > > I did try some other AWS instance types, but anything with 2GB or less > > failed due to out of memory issues. I'd be interested to try some other > > types, particularly the AMD-based types. > > > > Anyway long story short: yes, our build could be faster with better > > hardware. Just not very clear what that hardware should be. > > > > > > -David > > > > > > > On Oct 11, 2022, at 3:05 PM, David Blevins <david.blev...@gmail.com> > > wrote: > > > > > > All, > > > > > > I'm collecting some stats on how long it takes to run our full build > > exactly as Jenkins does. The goal is to work with them to see if we can > > get some better hardware -- I assume that will require donations, etc. > > > > > > If you'd like to help in collecting data, here's the script I'm > running: > > > > > > - curl > > > https://gist.githubusercontent.com/dblevins/b39cc3300bcdd89b426ca33b87b5452b/raw/7c68d4df71e9246c8bf2d0a741f8b145ca5d0820/buildtime.sh > > | bash > > > > > > Send the time reported in the build.log along with your system > > information (os, number of cores, if you disk is an SSD, etc) > > > > > > > > > -- > > > David Blevins > > > http://twitter.com/dblevins > > > http://www.tomitribe.com > > > > > > > > -- Atentamente: César Hernández.
