> > > The risk is caching a negative result coming from a flaky test. And I > feel > like James test suite is quite unstable. > > On that, maybe we can find if the Gradle extension / surefire supports > caching only if the build (test phase for example) succeeds? > > I searched the documentation again ( https://docs.gradle.com/enterprise/maven-extension/#maven_surefire_plugin_and_maven_failsafe_plugin ) > Test results for surefire:test are stored in the Build Cache whenever the goal succeeds. Thus, by default, only successful or skipped test results are cached. However, if <testFailureIgnore> <https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testFailureIgnore> is set to true, test failures are cached as well.
And good news <testFailureIgnore> is not used in the james build so I can´t see a reason not to enable remote caching. jean Quan > > Vào Th 4, 7 thg 2, 2024 vào lúc 10:22 Rene Cordier <rcord...@apache.org> > đã viết: > > > I agree caching the test suite is very tempting, it would help gain > > sooooo much time on the build. > > > > But yes the test suite is flaky unfortunately. Maybe we need extra work > > making it more stable first, but that's not easy task ^^' > > > > Or maybe for remote caching on test execution, maybe we can try to > > execute that in an other branch in parallel to master, giving us time to > > monitor the behavior of the build, before using it for good? Requires > > extra work, efforts and attention yes, but I hardly see how else we can > > go safely forward on this. But somebody else has a better idea maybe? :) > > > > Regards, > > > > Rene. > > > > On 2/7/24 02:58, Jean Helou wrote: > > > Thanks for trying > > > > > > As you can see in the output only about 10% of the tasks you ran were > > > cached. > > > > > > Which explains why the gains are not so great. > > > > > > There are 2 good news > > > - it is possible to configure more goals to be cached. I have an > attempt > > at > > > caching scala compile (which doesn't work for now as that goal has > rather > > > complex input and outputs). > > > I intend to look at build scans to cache more high CPU consuming goals > > and > > > to avoid running some goals entirely when possible (as per my comments > on > > > the PR) > > > > > > Where it becomes really interesting (the second good news) is that > > surefire > > > is supported out of the box, so if one pays the cost of running the > > tests, > > > every subsequent build would only run the tests for the changed > projects > > > and their dependencies > > > > > > And testing is by far the most time consuming goal in James build. > > > > > > Remote caching of test execution would have the most impact, > accelerating > > > both local and CI builds because it would allow sharing the cache > between > > > different machines/nodes. (Only the ci would write but everyone could > > read > > > if I understand correctly) > > > > > > The risk is caching a negative result coming from a flaky test. And I > > feel > > > like James test suite is quite unstable. > > > > > > So I'm wondering if I should go ahead with the remote build cache node > > > request to apache INFRA or revisit that request in a few months. > > > > > > Jean > > > > > > > > > > > > Le mar. 6 févr. 2024 à 11:18, Quan tran hong < > > quan.tranhong1...@gmail.com> > > > a écrit : > > > > > >> Hi Jean, > > >> > > >> I tried your work locally and the build time is truly better. > > >> `mvn clean install -DskipTests -Dmaven.skip.doc=true > > >> -Dgradle.cache.local.enabled=true` > > >> > > >> 1st build: > > >> [INFO] Total time: 22:18 min > > >> [INFO] 8053 goals, 7996 executed, 57 from cache, saving at least 13s > > >> > > >> 2nd build: > > >> [INFO] Total time: 15:07 min > > >> [INFO] 8053 goals, 7296 executed, 757 from cache, saving at least 3m > 11s > > >> > > >> Thank you for the initiative :-) > > >> > > >> Quan > > >> > > >> Vào Th 3, 6 thg 2, 2024 vào lúc 14:48 Jean Helou <jhe...@apache.org > > > > đã > > >> viết: > > >> > > >>> Hi devs, > > >>> > > >>> I realized this morning that I initiated the conversation on the > wrong > > >> list > > >>> (server-user instead of server-dev) ... > > >>> see the archives > > >>> https://lists.apache.org/list.html?server-u...@james.apache.org for > > >> Rene's > > >>> positive answer > > >>> > > >>> sorry about that > > >>> > > >>> jean > > >>> > > >>> ---------- Forwarded message --------- > > >>> From: Jean Helou <jhe...@apache.org> > > >>> Date: Mon, Feb 5, 2024 at 3:52 PM > > >>> Subject: Develocity > > >>> To: James Users List <server-u...@james.apache.org> > > >>> > > >>> > > >>> Hello fellow james devs, > > >>> > > >>> I'm not sure how much context to provide in the email you can read > more > > >> in > > >>> > > >> > > > https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3978?filter=allopenissues > > >>> TLDR; A few months ago the apache foundation announced a sponsorship > of > > >> the > > >>> "gradle" company for a tool to diagnose, monitor and optimize > builds ( > > >>> called gradle enterprise at the time and renamed develocity since) > > >>> > > >>> I posted a first PR to plug james build in that tool > > >>> https://github.com/apache/james-project/pull/1972 > > >>> > > >>> This first integration brings only build scans and local caching > > >>> - build scans can help us understand what slows our build > > >>> - local caching allows local build perf improvements (I posted a > small > > >>> benchmark which shows a divide by 3 the time spent building > > >>> james-server-core in a repeat build with no changes) > > >>> > > >>> The build scans also show that many tasks in the build are not > actually > > >>> cached so there is room for more improvement. > > >>> > > >>> The local cache does apply to surefire tasks,unfortunately the > benefits > > >> for > > >>> CI are very small : > > >>> > > >>> In the first stage we cache the compilation result itself but that > > >>> represents a very small part of the test stages (even if we had to > > >> compile > > >>> everything from scratch that's barely 20 minutes out of the 3h30 of > the > > >>> build so caching only marginally improves the build time at the > moment > > >>> > > >>> the next stage would be to : > > >>> - make more tasks cacheable > > >>> - enable the remote build cache > > >>> > > >>> The remote build cache allows to share caching accross builds which > > means > > >>> that if a change affects only a small portion of the build, only that > > >>> change would effectively be recompiled and retested, so the closer to > > the > > >>> "leafs" of the reactor a change would be the shorter its CI pipelines > > >> would > > >>> be. > > >>> I pinged INFRA in https://issues.apache.org/jira/browse/INFRA-25461 > to > > >>> know > > >>> what the process is for enabling the remote build cache and it seems > it > > >>> starts with a mailing list thread :D > > >>> > > >>> note that > > >>> - only the CI would be allowed to write to the remote build cache > > >>> - build caches both local and remote can be disabled with properties > > >> (IIRC > > >>> there are pretty strict constraints on release processes in the > apache > > >>> foundation) > > >>> - I think it is possible to have the remote build cache be activated > in > > >>> readonly mode for everyone so that anyone pulling the repo would > > >>> experience a very fast first build (but that remains to be confirmed > > as I > > >>> have never experienced it) > > >>> > > >>> what do you think ? > > >>> > > >>> jean > > >>> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > > For additional commands, e-mail: server-dev-h...@james.apache.org > > > > >
