Just want to say +1, and thanks for taking the time to do this. This all sounds great.
(And completely supersedes what I hoped to achieve with CASSANDRA-14648) > On 26 Oct 2018, at 15:49, Stefan Podkowinski <s...@apache.org> wrote: > > I'd like to give you a quick update on the work that has been done > lately on running tests using CircleCI. Please let me know if you have > any objections or don't think this is going into the right direction, or > have any other feedback! > > We've been using CircleCI for a while now and results are used on > constant basis for new patches. Not only by committers, but also by > casual contributors to run unit tests. Looks like people find the > service valuable and we should keep using it. Therefor I'd like to make > some improvements that will make it easier to add new tests and to > continue making CircleCI an option for all contributors, both on paid > and free plans. > > The general idea of the changes implemented in #14806, is to consolidate > the existing config to make it more modular and have smaller jobs that > can be scheduled ad-hoc by the developer, instead of running a few big > jobs on every commit. Reorganizing and breaking up the existing config > was done using the new 2.1 config features. Starting jobs on request, > instead of automatically, is done using the manual approval feature, > i.e. you now have to click on that job in the workflow page in order to > start it. I'd like to see us having smaller, more specialized groups of > tests that we can run more selectively during development, while still > being able to run bigger tests before committing, or firing up all of > them during testing and releasing. Other example of smaller jobs would > be testing coverage (#14788) or cqlsh tests (#14298). But also > individual jobs for different ant targets, like burn, stress or benchmarks. > > We'd now also be able to run tests using different docker images and > different JDKs. I've already updated the used image to also include Java > 11 and added unit and dtest jobs to the config for that. It's now really > easy to run tests on Java 11, although these won't pass yet. It seems to > be important to me to have this kind of flexibility, given the > increasingly diverse ecosystem of Java distributions. We can also add > jobs for packaging and doing smoke tests by installing and starting > packages on different docker images (Redhat, Debian, Ubuntu,..) at a > later point. > > As for the paid vs free plans issue, I'd also like us to discuss how we > can make tests faster and less resource intensive in general. As a > desired consequence, we'd be able to move away from multi-node dtests, > to something that can be run using the free plan. I'm looking forward to > see if #14821 can get us into that direction. Ideally we can add these > tests into a job that can be completed on the free plan and encourage > contributors to add new tests there, instead of having to write a dtest, > which they won't be able to run on CircleCI without a paid plan. > > Whats changing for you as a CircleCI user? > * All tests, except unit tests, will need to be started manually and > will not run on every commit (this can be further discussed and changed > anytime if needed) > * Updating the config.yml file now requires using the CircleCI cli tool > and should not be done directly (see #14806 for technical details) > * High resource settings can be enabled using a script/patch, either run > manually or as commit hook (again see ticket for details) > * Both free and paid plan users now have more tests to run > > As already mentioned, please let me know if you have any thoughts on > this, or if you think this is going into the wrong direction. > > Thanks. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
