And one more thing - how much time is spent on duplicated "Binary" suites? *15 hours*, or almost *40%* of total "Run All" execution time. It means, that if move suites for *OptimizedMarshaller *(which we consider deprecated) into a separate circuit (which will be executed not per-branch, but e.g. nightly) we will almost double peak TeamCity throughput.
On Fri, Dec 16, 2016 at 6:23 PM, Vladimir Ozerov <[email protected]> wrote: > Igniters, > > I collected several thought-provoking numbers for our TeamCity > (config-variations tests are excluded): > > * Number of Java suites: 78 > * Total execution time without build overhead: 38.5 hours > * Total test classes: 3376 > * Total tests: 60681 > > * 25% of time is consumed by 881 tests (41 test class) > * 50% of time is consumed by 2755 tests (153 test class) > * 75% of time is consumed by 15911 tests (443 test class) > > Or to be more demonstrative: > * 1.5% of tests consume 25% of time > * 4.5% of tests consume 50% of time > * 26.2% tests consume 75% of time > > TOP 3 test classes: > * CacheSerializableTransactionsTest - 46 minutes > * IgniteCacheCrossCacheTxFailoverTest - 30 minutes > * GridCachePartitionedNodeRestartTest - 25 minutes > > Morale: > 1) We have lots of poor failover/concurrency tests requiring > disproportional time to run; > 2) If we split all tests into "core" and "failover/concurrency" parts > properly, then *core* part will take *~15 mins* to answer you with high > level of confidence whether your change broken anything or not. > > Thoughts? > > Vladimir. >
