[
https://issues.apache.org/jira/browse/JAMES-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823786#comment-17823786
]
Jean Helou commented on JAMES-3978:
-----------------------------------
a formal vote has validated an experimentation enabling develocity remote build
cache for james builds.In the mean time we now have about a month of metrics
and build scans and all.
The metrics are mostly about CI as the capture of build scan has not been
enabled systematically, therefore the local build cache gains are not very
visible. However the insight brought by the build scans has been translated in
a few improvements on the build configuration reducing the median time of the
CI build stage a bit (see
[https://ge.apache.org/scans/trends?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.tags=master&search.tasks=clean%2Cinstall&search.timeZoneId=Europe%2FParis)]
For the local build cache, I still haven't been able to properly configure
caching for scala compilation, or scala typecheck which are both quite
expensive.
As reported on the mailing list, enably flakyness detection enables a deeper
dive into james tests :
[https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.tags=master&search.tasks=test%2Cjacoco:report-aggregate@jacoco-report&search.timeZoneId=Europe%2FParis&tests.sortField=FLAKY#]
it seems that
[org.apache.james.transport.mailets.RemoteDeliveryErrorHandlingTest|https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.tags=master&search.tasks=test%2Cjacoco:report-aggregate@jacoco-report&search.timeZoneId=Europe%2FParis&tests.container=org.apache.james.transport.mailets.RemoteDeliveryErrorHandlingTest&tests.sortField=FLAKY]
is especially flaky both on master and accross all PRs
cassandra based integration tests are also quite flaky, with quite a few
failures linked to
```
[ com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out
after
PT2S|https://ge.apache.org/s/uqvq75ttvkrs6/tests/goal/org.apache.james:blob-cassandra:surefire:test@default-test/details/org.apache.james.blob.cassandra.CassandraBlobStoreClOneTest/blobStoreShouldSupport100MBBlob]
```
> Investigate using develocity to improve james build
> ---------------------------------------------------
>
> Key: JAMES-3978
> URL: https://issues.apache.org/jira/browse/JAMES-3978
> Project: James Server
> Issue Type: Improvement
> Components: Build System
> Reporter: Jean Helou
> Priority: Major
> Attachments: image-2024-02-02-16-30-20-026.png,
> image-2024-02-02-16-31-24-883.png
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Hello,
> A while ago I noticed [this
> announcement|https://news.apache.org/foundation/entry/the-apache-software-foundation-announces-gradle-as-a-platinum-targeted-sponsor]
> gradle becoming a platinum sponsor of the ASF.
> In particular the announcement was about gradle offering the develocity
> service to the ASF. It so happens that I investigated develocity as part of
> my day job, and while we can't afford it there it is definitely something we
> might leverage here !
> The tool is available at [https://ge.apache.org|https://ge.apache.org/]
> I want to investigate the following leads
>
> # local caching for builds enable reuse of tasks output to speed up repeated
> tasks
> # build scans to provide detailed build reports and individual build metrics
> # build metrics are collected across all build scans and allow to target
> objectively bad contributors to overall build time
> # enabling the remote build cache
> ** additionally improves both local and CI build times
> ** requires some attention to read/write permissions to avoid cache
> poisoning and supply chain attacks
> ** most likely requires a dedicated build cache node request to infra
> ** several projects already use it so it is a known unknown ( we know it can
> be done we don't yet know how)
> # leverage advanced test features of develocity unknown unknowns at least
> for me:D
> ## the tool collects tests metrics to pinpoint flaky tests enabling targeted
> fix efforts
> ## the tool claims to be able to do test avoidance and not rerun tests that
> are irrelevant to the changes
> ## the tool claims to be able to distribute test excecution accross
> dedicated test workers
> Since we use maven, we won't get the full benefits of the build cache (
> gradle has been pushing plugin authors to write build cache friendly code for
> a long time ) but it should improve the current situation quite a bit.
>
> Apache pulsar is also using develocity with maven:
> - develocity configuration
> [https://github.com/apache/pulsar/blob/master/.mvn/gradle-enterprise.xml]
> - infra wiki has infos on how to onboard
> https://cwiki.apache.org/confluence/display/INFRA/Project+Onboarding+Instructions+for+Develocity
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
