Hi Igniters! I made a research into tests that aren't included in any test suite. As TeamCity runs tests by suites so there could be tests that never run on TC. So I tried implementing a simple check for such tests and include it in Ignite's travis config.
The check runs while "mvn test" command and piggy-backs on the maven surefire plugin. I replaced the junit provider with a custom one that checks if a class is a test or a suite (there are some Ignite specific stuff), marks tests that are in suites and raises an exception if there are non-suited tests. It's implemented as a part of maven command so it runs for every module separately. I've prepared draft PR with this check: https://github.com/apache/ignite/pull/8367 Travis check report is here: https://travis-ci.org/github/apache/ignite/jobs/737046387 As It's a draft, so I skip some maven configuration steps for a while. Also I run the check only for the core module. But I have some results that want to discuss before continue the work: 1. Currently in the core module there are 53 tests that aren't part of any test suite. I'm not sure about the reason for every test. So I just put below a list of the tests and last contributor to a file that contains a test. 2. Some tests are located in the core module, but suites are in a different, for example ignite-indexing suite IgniteCacheQuerySelfTestSuite3 contains only tests written in the core module, and none from the indexing module. Also there are suites in spring, uri-deploy, zookeeper modules. In my PR I've just copied the test suites to the core module. 3. Some test classes are named with the "Abstract" suffix but don't have the corresponding modifier (for example, IgniteTxTimeoutAbstractTest). So, I add the modifier for every such file if it's not a part of any suite. What do you think about this check? If Ignite needs it, let's discuss next things: 1. Mark tests that should never be in any suite by some reason; 2. Fix the missed tests; 3. How to declare suites that contains tests from a different module; 4. How to check if TC runs all suites. List of non-suited tests in the core module: maksim.stepac...@gmail.com: GridTcpCommunicationSpiLogTest nizhi...@apache.org: IgniteCacheClientMultiNodeUpdateTopologyLockTest CacheClientsConcurrentStartTest IgniteOutOfMemoryPropagationTest GridCacheP2PUndeploySelfTest GridCacheRebalancingOrderingTest IgniteMassLoadSandboxTest PageLockTrackerMXBeanImplTest IgniteBinaryMetadataUpdateNodeRestartTest CacheLockCandidatesThreadTest GridMBeanBaselineTest RendezvousAffinityFunctionSimpleBenchmark samvi...@yandex.ru: IgnitePdsNoSpaceLeftOnDeviceTest maxmu...@gmail.com: GridCacheOnCopyFlagReplicatedSelfTest GridCacheOnCopyFlagLocalSelfTest GridCacheReplicatedAtomicReferenceMultiNodeTest GridCacheReplicatedMarshallerTxTest GridCacheReplicatedTxConcurrentGetTest GridCacheOnCopyFlagTxPartitionedSelfTest GridCacheReplicatedTxReadTest GridCachePartitionedAtomicReferenceMultiNodeTest GridCacheOnCopyFlagAtomicSelfTest mmu...@apache.org: GridActivateExtensionTest IgniteChangeGlobalStateCacheTest IgniteChangeGlobalStateTest IgniteChangeGlobalStateServiceTest IgniteChangeGlobalStateDataStructureTest oignate...@gridgain.com: CacheEntryProcessorCopySelfTest MemoryLeaksOnRestartNodeTest GridCacheAtomicPreloadSelfTest WalCompactionAfterRestartTest IgniteCacheConcurrentPutGetRemove GridIoManagerBenchmark0 nsamelc...@gmail.com: GridLongRunningInitNewCrdFutureDiagnosticsTest GridCacheMultithreadedFailoverAbstractTest alexey.goncha...@gmail.com: GridCacheBinaryObjectsAtomicOnheapSelfTest GridCacheBinaryObjectsAtomicNearDisabledOnheapSelfTest GridCacheBinaryObjectsPartitionedOnheapSelfTest GridCacheBinaryObjectsPartitionedNearDisabledOnheapSelfTest vladis...@gmail.com: IgnitePartitionedLockSelfTest alexandr.bel...@xored.com: IgniteStableBaselineCachePutAllFailoverTest IgniteStableBaselineCacheRemoveFailoverTest ilant...@gridgain.com: IgniteCacheAtomicOnheapExpiryPolicyTest IgniteCacheAtomicLocalOnheapExpiryPolicyTest GridCacheReplicatedOnheapFullApiSelfTest GridCacheBinaryObjectsLocalOnheapSelfTest oignate...@users.noreply.github.com: GridCacheTtlManagerEvictionSelfTest ira...@apache.org: CommonPoolStarvationCheckpointTest alievmi...@gmail.com: RemoveAllDeadlockTest schugu...@gridgain.com: FullyConnectedComponentSearcherTest sboi...@gridgain.com: IgniteDataStructuresNoClassOnServerTest timonin.ma...@gmail.com: ReliableChannelTest ThinClientPartitionAwarenessDiscoveryTest
