Nobody referred so far to the idea of moving to JUnit 5, what are the opinions?
niedz., 10 gru 2023 o 11:03 Benedict <bened...@apache.org> napisał(a): > Alex’s suggestion was that we meta randomise, ie we randomise the config > parameters to gain better rather than lesser coverage overall. This means > we cover these specific configs and more - just not necessarily on any > single commit. > > I strongly endorse this approach over the status quo. > > On 8 Dec 2023, at 13:26, Mick Semb Wever <m...@apache.org> wrote: > > > > > >> >> I think everyone agrees here, but…. these variations are still catching >>> failures, and until we have an improvement or replacement we do rely on >>> them. I'm not in favour of removing them until we have proof /confidence >>> that any replacement is catching the same failures. Especially oa, tries, >>> vnodes. (Not tries and offheap is being replaced with "latest", which >>> will be valuable simplification.) >> >> >> What kind of proof do you expect? I cannot imagine how we could prove >> that because the ability of detecting failures results from the randomness >> of those tests. That's why when such a test fail you usually cannot >> reproduce that easily. >> > > > Unit tests that fail consistently but only on one configuration, should > not be removed/replaced until the replacement also catches the failure. > > > >> We could extrapolate that to - why we only have those configurations? why >> don't test trie / oa + compression, or CDC, or system memtable? >> > > > Because, along the way, people have decided a certain configuration > deserves additional testing and it has been done this way in lieu of any > other more efficient approach. > > >
