Hi fellows, The Develocity <https://ge.apache.org> integration of james-project build has now been running for long enough that we can start seeing interesting things beyond the raw build scans. First we can give a short look at build trends <https://ge.apache.org/scans/trends?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.tags=master&search.tasks=clean%2Cinstall&search.timeZoneId=Europe%2FParis#> for the `clean install` command on master (the build stage), its hard to derive actual trends since - we only have 15 days of history - we run on runners with varying cpu power ( build scans show T1C go from 16 to 24) but it can be used to establish a baseline so we can compare later on In average this stage took 9min10s the week from feb 5 to feb 11th has had a lot more spread 5th-95th percentile was 4m54s to 11m26s last week was a bit more homogenous with the same percentiles being 7m54 - 10m18 this week so far looks more similar to last week than the previous, 7m12 - 9m4
Adding the local build cache added about 14s of overhead and provided no benefits on CI (completely expected, maybe I should disable local caching on CI since it spawns in a fresh env everytime) but this will be interesting to compare with the remote build cache. The tests <https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.timeZoneId=Europe%2FParis&tests.sortField=FLAKY> monitoring is also interesting. This screen monitors 2 things: - which tests fail the most often - which tests are flaky To measure this I enabled retries for surefire in this commit <https://github.com/apache/james-project/commit/39a50194> so that a failing test is retried once. Failed is both tries failed Flakyness is derived by looking at retries (1 fail followed by 1 success). The results probably won't surprise you too much : The test suite with most failures (11% of the runs) is org.apache.james.blob.cassandra.CassandraBlobStoreTest <https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.timeZoneId=Europe%2FParis&tests.container=org.apache.james.blob.cassandra.CassandraBlobStoreTest> The test suite with most flakyness (6% of the runs) is org.apache.james.blob.cassandra.CassandraBlobStoreDAOTest <https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.timeZoneId=Europe%2FParis&tests.container=org.apache.james.blob.cassandra.CassandraBlobStoreDAOTest&tests.sortField=FLAKY> The longest running testsuite is org.apache.james.backends.rabbitmq.RabbitMQTest <https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=*james*&search.timeZoneId=Europe%2FParis&tests.container=org.apache.james.backends.rabbitmq.RabbitMQTest&tests.sortField=MEAN_DURATION> at 4min27s Top ten failing test suites org.apache.james.blob.cassandra.CassandraBlobStoreTest org.apache.james.blob.cassandra.cache.CachedBlobStoreTest org.apache.james.blob.cassandra.CassandraBucketDAOTest org.apache.james.blob.cassandra.CassandraDefaultBucketDAOTest org.apache.james.blob.cassandra.CassandraPassTroughBlobStoreTest org.apache.james.blob.cassandra.CassandraBlobStoreClOneTest org.apache.james.blob.cassandra.CassandraBlobStoreDAOTest org.apache.james.blob.cassandra.cache.CassandraBlobStoreCacheTest org.apache.james.blob.objectstorage.aws.S3MinioTest org.apache.james.blob.objectstorage.aws.S3BlobStoreDAOTest Top ten flaky doesn't completely overlap org.apache.james.blob.cassandra.CassandraBlobStoreDAOTest org.apache.james.user.ldap.ReadOnlyUsersLDAPRepositoryWithLocalPartAsLoginNameTest$WhenEnableVirtualHosting$LocalPartLogin org.apache.james.user.ldap.ReadOnlyUsersLDAPRepositoryWithLocalPartAsLoginNameTest$WhenEnableVirtualHosting$OneAppPasswordLogin org.apache.james.user.ldap.ReadOnlyUsersLDAPRepositoryTest$WhenDisableVirtualHosting org.apache.james.blob.cassandra.CassandraBlobStoreTest org.apache.james.user.ldap.ReadOnlyUsersLDAPRepositoryWithLocalPartAsLoginNameTest$WhenEnableVirtualHosting org.apache.james.blob.cassandra.CassandraPassTroughBlobStoreTest org.apache.james.user.ldap.LdapHealthCheckTest org.apache.james.user.ldap.ReadOnlyUsersLDAPRepositoryTest$WhenEnableVirtualHosting org.apache.james.webadmin.integration.rabbitmq.vault.RabbitMQDeletedMessageVaultIntegrationTest you can follow all the details on the apache develocity :D Cheers jean
