Hello Jean, I think this is a great initiative, thank you!
The build are regularly unstable and painful to maintain, such tooling to maybe make builds faster, not rebuilding and retesting everything every time for small changes and scans to understand better about where we spend build time sound like very good news to me.
A big +1 on this :) Rene. On 2/5/24 21:52, Jean Helou wrote:
Hello fellow james devs, I'm not sure how much context to provide in the email you can read more in https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3978?filter=allopenissues TLDR; A few months ago the apache foundation announced a sponsorship of the "gradle" company for a tool to diagnose, monitor and optimize builds ( called gradle enterprise at the time and renamed develocity since) I posted a first PR to plug james build in that tool https://github.com/apache/james-project/pull/1972 This first integration brings only build scans and local caching - build scans can help us understand what slows our build - local caching allows local build perf improvements (I posted a small benchmark which shows a divide by 3 the time spent building james-server-core in a repeat build with no changes) The build scans also show that many tasks in the build are not actually cached so there is room for more improvement. The local cache does apply to surefire tasks,unfortunately the benefits for CI are very small : In the first stage we cache the compilation result itself but that represents a very small part of the test stages (even if we had to compile everything from scratch that's barely 20 minutes out of the 3h30 of the build so caching only marginally improves the build time at the moment the next stage would be to : - make more tasks cacheable - enable the remote build cache The remote build cache allows to share caching accross builds which means that if a change affects only a small portion of the build, only that change would effectively be recompiled and retested, so the closer to the "leafs" of the reactor a change would be the shorter its CI pipelines would be. I pinged INFRA in https://issues.apache.org/jira/browse/INFRA-25461 to know what the process is for enabling the remote build cache and it seems it starts with a mailing list thread :D note that - only the CI would be allowed to write to the remote build cache - build caches both local and remote can be disabled with properties (IIRC there are pretty strict constraints on release processes in the apache foundation) - I think it is possible to have the remote build cache be activated in readonly mode for everyone so that anyone pulling the repo would experience a very fast first build (but that remains to be confirmed as I have never experienced it) what do you think ? jean
--------------------------------------------------------------------- To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org For additional commands, e-mail: server-user-h...@james.apache.org
