> that may be long-running and that could be run indefinitely Perfect. That was the distinction I wasn't aware of. Also means having the burn target as part of regular CI runs is probably a mistake, yes? i.e. if someone adds a burn tests that runs indefinitely, are there any guardrails or built-in checks or timeouts to keep it from running right up to job timeout and then failing?
On Thu, Nov 30, 2023, at 1:11 PM, Benedict wrote: > > A burn test is a randomised test targeting broad coverage of a single system, > subsystem or utility, that may be long-running and that could be run > indefinitely, each run providing incrementally more assurance of quality of > the system. > > A long test is a unit test that sometimes takes a long time to run, no more > no less. I’m not sure any of these offer all that much value anymore, and > perhaps we could look to deprecate them. > >> On 30 Nov 2023, at 17:20, Josh McKenzie <jmcken...@apache.org> wrote: >> >> Strongly agree. I started working on a declarative refactor out of our CI >> configuration so circle, ASFCI, and other systems could inherit from it (for >> instance, see pre-commit pipeline declaration here >> <https://github.com/apache/cassandra/pull/2554/files#diff-a4c4d1d91048841f76d124386858bda9944644cfef8ccb4ab84319cedaf5b3feR71-R89>); >> I had to set that down while I finished up implementing an internal CI >> system since the code in neither the ASF CI structure nor circle structure >> (.sh embedded in .yml /cry) was re-usable in their current form. >> >> Having a jvm.options and cassandra.yaml file per suite and referencing them >> from a declarative job definition >> <https://github.com/apache/cassandra/pull/2554/files#diff-a4c4d1d91048841f76d124386858bda9944644cfef8ccb4ab84319cedaf5b3feR237-R267> >> would make things a lot easier to wrap our heads around and maintain I >> think. >> >> As for what qualifies as burn vs. long... /shrug couldn't tell you. Would >> have to go down the git blame + dev ML + JIRA rabbit hole. :) Maybe someone >> else on-list knows. >> >> On Thu, Nov 30, 2023, at 4:25 AM, Jacek Lewandowski wrote: >>> Hi, >>> >>> I'm getting a bit lost - what are the exact differences between those test >>> scenarios? What are the criteria for qualifying a test to be part of a >>> certain scenario? >>> >>> I'm working a little bit with tests and build scripts and the number of >>> different configurations for which we have a separate target in the build >>> starts to be problematic, I cannot imagine how problematic it is for a new >>> contributor. >>> >>> It is not urgent, but we should at least have a plan on how to simplify and >>> unify things. >>> >>> I'm in favour of reducing the number of test targets to the minimum - for >>> different configurations I think we should provide a parameter pointing to >>> jvm options file and maybe to cassandra.yaml. I know that we currently do >>> some super hacky things with cassandra yaml for different configs - like >>> concatenting parts of it. I presume it is not necessary - we can have a >>> default test config yaml and a directory with overriding yamls; while >>> building we could have a tool which is able to load the default >>> configuration, apply the override and save the resulting yaml somewhere in >>> the build/test/configs for example. That would allows us to easily use >>> those yamls in IDE as well - currently it is impossible. >>> >>> What do you think? >>> >>> Thank you and my apologize for bothering about lower priority stuff while >>> we have a 5.0 release headache... >>> >>> Jacek >>> >>
