Hi, > The risk is caching a negative result coming from a flaky test. And I feel like James test suite is quite unstable.
On that, maybe we can find if the Gradle extension / surefire supports caching only if the build (test phase for example) succeeds? Quan Vào Th 4, 7 thg 2, 2024 vào lúc 10:22 Rene Cordier <rcord...@apache.org> đã viết: > I agree caching the test suite is very tempting, it would help gain > sooooo much time on the build. > > But yes the test suite is flaky unfortunately. Maybe we need extra work > making it more stable first, but that's not easy task ^^' > > Or maybe for remote caching on test execution, maybe we can try to > execute that in an other branch in parallel to master, giving us time to > monitor the behavior of the build, before using it for good? Requires > extra work, efforts and attention yes, but I hardly see how else we can > go safely forward on this. But somebody else has a better idea maybe? :) > > Regards, > > Rene. > > On 2/7/24 02:58, Jean Helou wrote: > > Thanks for trying > > > > As you can see in the output only about 10% of the tasks you ran were > > cached. > > > > Which explains why the gains are not so great. > > > > There are 2 good news > > - it is possible to configure more goals to be cached. I have an attempt > at > > caching scala compile (which doesn't work for now as that goal has rather > > complex input and outputs). > > I intend to look at build scans to cache more high CPU consuming goals > and > > to avoid running some goals entirely when possible (as per my comments on > > the PR) > > > > Where it becomes really interesting (the second good news) is that > surefire > > is supported out of the box, so if one pays the cost of running the > tests, > > every subsequent build would only run the tests for the changed projects > > and their dependencies > > > > And testing is by far the most time consuming goal in James build. > > > > Remote caching of test execution would have the most impact, accelerating > > both local and CI builds because it would allow sharing the cache between > > different machines/nodes. (Only the ci would write but everyone could > read > > if I understand correctly) > > > > The risk is caching a negative result coming from a flaky test. And I > feel > > like James test suite is quite unstable. > > > > So I'm wondering if I should go ahead with the remote build cache node > > request to apache INFRA or revisit that request in a few months. > > > > Jean > > > > > > > > Le mar. 6 févr. 2024 à 11:18, Quan tran hong < > quan.tranhong1...@gmail.com> > > a écrit : > > > >> Hi Jean, > >> > >> I tried your work locally and the build time is truly better. > >> `mvn clean install -DskipTests -Dmaven.skip.doc=true > >> -Dgradle.cache.local.enabled=true` > >> > >> 1st build: > >> [INFO] Total time: 22:18 min > >> [INFO] 8053 goals, 7996 executed, 57 from cache, saving at least 13s > >> > >> 2nd build: > >> [INFO] Total time: 15:07 min > >> [INFO] 8053 goals, 7296 executed, 757 from cache, saving at least 3m 11s > >> > >> Thank you for the initiative :-) > >> > >> Quan > >> > >> Vào Th 3, 6 thg 2, 2024 vào lúc 14:48 Jean Helou <jhe...@apache.org> > đã > >> viết: > >> > >>> Hi devs, > >>> > >>> I realized this morning that I initiated the conversation on the wrong > >> list > >>> (server-user instead of server-dev) ... > >>> see the archives > >>> https://lists.apache.org/list.html?server-u...@james.apache.org for > >> Rene's > >>> positive answer > >>> > >>> sorry about that > >>> > >>> jean > >>> > >>> ---------- Forwarded message --------- > >>> From: Jean Helou <jhe...@apache.org> > >>> Date: Mon, Feb 5, 2024 at 3:52 PM > >>> Subject: Develocity > >>> To: James Users List <server-u...@james.apache.org> > >>> > >>> > >>> Hello fellow james devs, > >>> > >>> I'm not sure how much context to provide in the email you can read more > >> in > >>> > >> > https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3978?filter=allopenissues > >>> TLDR; A few months ago the apache foundation announced a sponsorship of > >> the > >>> "gradle" company for a tool to diagnose, monitor and optimize builds ( > >>> called gradle enterprise at the time and renamed develocity since) > >>> > >>> I posted a first PR to plug james build in that tool > >>> https://github.com/apache/james-project/pull/1972 > >>> > >>> This first integration brings only build scans and local caching > >>> - build scans can help us understand what slows our build > >>> - local caching allows local build perf improvements (I posted a small > >>> benchmark which shows a divide by 3 the time spent building > >>> james-server-core in a repeat build with no changes) > >>> > >>> The build scans also show that many tasks in the build are not actually > >>> cached so there is room for more improvement. > >>> > >>> The local cache does apply to surefire tasks,unfortunately the benefits > >> for > >>> CI are very small : > >>> > >>> In the first stage we cache the compilation result itself but that > >>> represents a very small part of the test stages (even if we had to > >> compile > >>> everything from scratch that's barely 20 minutes out of the 3h30 of the > >>> build so caching only marginally improves the build time at the moment > >>> > >>> the next stage would be to : > >>> - make more tasks cacheable > >>> - enable the remote build cache > >>> > >>> The remote build cache allows to share caching accross builds which > means > >>> that if a change affects only a small portion of the build, only that > >>> change would effectively be recompiled and retested, so the closer to > the > >>> "leafs" of the reactor a change would be the shorter its CI pipelines > >> would > >>> be. > >>> I pinged INFRA in https://issues.apache.org/jira/browse/INFRA-25461 to > >>> know > >>> what the process is for enabling the remote build cache and it seems it > >>> starts with a mailing list thread :D > >>> > >>> note that > >>> - only the CI would be allowed to write to the remote build cache > >>> - build caches both local and remote can be disabled with properties > >> (IIRC > >>> there are pretty strict constraints on release processes in the apache > >>> foundation) > >>> - I think it is possible to have the remote build cache be activated in > >>> readonly mode for everyone so that anyone pulling the repo would > >>> experience a very fast first build (but that remains to be confirmed > as I > >>> have never experienced it) > >>> > >>> what do you think ? > >>> > >>> jean > >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > For additional commands, e-mail: server-dev-h...@james.apache.org > >
