Hi, I think we should support something that we can scale out and that can produce consistent reproducible results.
One of the longer term issues is going to be different CI systems producing different sets of failures. It's not going to happen often but it will and it's one way flakiness will sneak in. There is an argument for settling on a single platform in that respect. If queue time are an issue and we can get more resources running multiple platforms then a different tradeoff will make sense. I have a desire to run my own CI box that can parallelize with containers, but I admit it's pretty niche use case and not worth much investment or overhead. Ariel On Tue, Mar 28, 2017, at 08:52 PM, Jeff Jirsa wrote: > To further our recent conversations on testing, I've opened the following > JIRA: > > https://issues.apache.org/jira/browse/CASSANDRA-13388 > > This should allow us to do SOME amount of automated testing for > developers > without jumping through a ton of hoops. If the free plans prove to be > insufficient, we can look into using the ASF's paid plan, or find other > solutions. > > If anyone has a reason that they feel this is inappropriate, speak up. > Otherwise, I'll try to put together a few yaml files this week to commit > soon'ish. > > Here's an example circle run that just kicks off unit tests on one of my > personal branches - https://circleci.com/gh/jeffjirsa/cassandra/35 - > nothing fancy, but at least it gives us the junit/testall run to show us > that we had one failure ( testDroppedMessages - > org.apache.cassandra.net.MessagingServiceTest ). It appears that we can > parallelize dtests and run dtests as a series of small tasks as well - we > may be able to get jacoco to run within the time limit, if we split up > unit > tests a bit. > > We can have this conversation on the list or on the JIRA, but does anyone > have a great reason why we SHOULDN'T support one or both of these > vendors, > or alternatively, is there another vendor we should consider instead? > > - Jeff
