michaelsembwever commented on code in PR #2554: URL: https://github.com/apache/cassandra/pull/2554#discussion_r1296363830
########## .build/config/README.md: ########## @@ -0,0 +1,125 @@ +Declarative Test Suite Configuration +------------------------------------------- + +Pipeline and test suite configurations are declarative so other CI implementations can build +durable, reactive systems based on changes to the upstream OSS C* CI. Additions to `jobs.cfg` and +`pipelines.cfg` can be picked up programmatically by CI implementations +without requiring human intervention. + +Concepts +--------------------- + +### Pipeline +A [pipeline](cassandra_ci.yaml) is a collection of jobs. For a given pipeline to be considered +successful, +all +jobs listed in the pipeline must run to completion without error using the constraints, commands, +and environment specified for the job in the config. + +### Job +A [job](jobs.yaml) contains a collection of parameters that inform a CI system on both what needs to +run, how to run it, and the constraints of the environment in which it should execute. We +provide these limits to reflect what's available in our reference ASF CI implementation so other +CI environments are able to limit themselves to our resourcing upstream and thus not destabilize +ASF CI. + +Examples of jobs include unit tests, python dtests, in-jvm dtests, etc. + +Jobs include the following parameters: + +* `parent:` Another job defined in the file this job inherits parameters from, potentially + overwriting any declared in duplication +* `description:` Text based description of this job's purpose +* `cmd:` The command a shell should run to execute the test job +* `testlist:` A command that will create a text file listing all the test files to be run for + this + suite +* `env:` Space delimited list of environment variables to be set for this suite. Duplicates for + params are allowed and later declarations should supercede former. +* `cpu:` Max cpu count allowed for a testing suite +* `memory:` Max memory (in GB) allowable for a suite +* `storage:` Max allowable storage (in GB) allowable for a suite to access + +Jobs can be split up and parallelized in whatever manner best suits the environment in which they're +orchestraed. + +Configuration Files +--------------------- + +[pipelines.cfg](./cassandra_ci.yaml): Contains pipelines for CI jobs for Apache Cassandra + +[jobs.cfg](./jobs.yaml): Contains reference CI jobs for Apache Cassandra + +Existing Pipelines +--------------------- + +As outlined in the `pipelines.cfg` file, we primarily have 3 pipelines: +### pre-commit: +* must run and pass on the lowest supported JDK before a committer merges any code +### post-commit: +* will run on the upstream ASF repo after a commit is merged, matrixed across more axes and including configurations expected to fail or diverge only rarely +### nightly: +* run nightly. Longer term, infra, very stable areas of code. Review Comment: do we need this yet? rationale: if post-commit doesn't have a turn-around time requirement, and costs are not consumption based, then we can run everything post-commit. we could also have sequential post-commit pipelines so feedback on the initial post-commit pipeline run is timely (which would essentially be similar to a nightly running repeatedly) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
