Thanks for trying As you can see in the output only about 10% of the tasks you ran were cached.
Which explains why the gains are not so great. There are 2 good news - it is possible to configure more goals to be cached. I have an attempt at caching scala compile (which doesn't work for now as that goal has rather complex input and outputs). I intend to look at build scans to cache more high CPU consuming goals and to avoid running some goals entirely when possible (as per my comments on the PR) Where it becomes really interesting (the second good news) is that surefire is supported out of the box, so if one pays the cost of running the tests, every subsequent build would only run the tests for the changed projects and their dependencies And testing is by far the most time consuming goal in James build. Remote caching of test execution would have the most impact, accelerating both local and CI builds because it would allow sharing the cache between different machines/nodes. (Only the ci would write but everyone could read if I understand correctly) The risk is caching a negative result coming from a flaky test. And I feel like James test suite is quite unstable. So I'm wondering if I should go ahead with the remote build cache node request to apache INFRA or revisit that request in a few months. Jean Le mar. 6 févr. 2024 à 11:18, Quan tran hong <quan.tranhong1...@gmail.com> a écrit : > Hi Jean, > > I tried your work locally and the build time is truly better. > `mvn clean install -DskipTests -Dmaven.skip.doc=true > -Dgradle.cache.local.enabled=true` > > 1st build: > [INFO] Total time: 22:18 min > [INFO] 8053 goals, 7996 executed, 57 from cache, saving at least 13s > > 2nd build: > [INFO] Total time: 15:07 min > [INFO] 8053 goals, 7296 executed, 757 from cache, saving at least 3m 11s > > Thank you for the initiative :-) > > Quan > > Vào Th 3, 6 thg 2, 2024 vào lúc 14:48 Jean Helou <jhe...@apache.org> đã > viết: > > > Hi devs, > > > > I realized this morning that I initiated the conversation on the wrong > list > > (server-user instead of server-dev) ... > > see the archives > > https://lists.apache.org/list.html?server-u...@james.apache.org for > Rene's > > positive answer > > > > sorry about that > > > > jean > > > > ---------- Forwarded message --------- > > From: Jean Helou <jhe...@apache.org> > > Date: Mon, Feb 5, 2024 at 3:52 PM > > Subject: Develocity > > To: James Users List <server-u...@james.apache.org> > > > > > > Hello fellow james devs, > > > > I'm not sure how much context to provide in the email you can read more > in > > > > > https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3978?filter=allopenissues > > > > TLDR; A few months ago the apache foundation announced a sponsorship of > the > > "gradle" company for a tool to diagnose, monitor and optimize builds ( > > called gradle enterprise at the time and renamed develocity since) > > > > I posted a first PR to plug james build in that tool > > https://github.com/apache/james-project/pull/1972 > > > > This first integration brings only build scans and local caching > > - build scans can help us understand what slows our build > > - local caching allows local build perf improvements (I posted a small > > benchmark which shows a divide by 3 the time spent building > > james-server-core in a repeat build with no changes) > > > > The build scans also show that many tasks in the build are not actually > > cached so there is room for more improvement. > > > > The local cache does apply to surefire tasks,unfortunately the benefits > for > > CI are very small : > > > > In the first stage we cache the compilation result itself but that > > represents a very small part of the test stages (even if we had to > compile > > everything from scratch that's barely 20 minutes out of the 3h30 of the > > build so caching only marginally improves the build time at the moment > > > > the next stage would be to : > > - make more tasks cacheable > > - enable the remote build cache > > > > The remote build cache allows to share caching accross builds which means > > that if a change affects only a small portion of the build, only that > > change would effectively be recompiled and retested, so the closer to the > > "leafs" of the reactor a change would be the shorter its CI pipelines > would > > be. > > I pinged INFRA in https://issues.apache.org/jira/browse/INFRA-25461 to > > know > > what the process is for enabling the remote build cache and it seems it > > starts with a mailing list thread :D > > > > note that > > - only the CI would be allowed to write to the remote build cache > > - build caches both local and remote can be disabled with properties > (IIRC > > there are pretty strict constraints on release processes in the apache > > foundation) > > - I think it is possible to have the remote build cache be activated in > > readonly mode for everyone so that anyone pulling the repo would > > experience a very fast first build (but that remains to be confirmed as I > > have never experienced it) > > > > what do you think ? > > > > jean > > >
