> > I think we'd be able to figure out the one of them causing a regression > on the day after.
That sounds great in theory. In practice, that doesn't happen unless one person steps up and makes themselves accountable for it. For reference, take a look at: https://cassci.datastax.com/view/trunk/, and https://issues.apache.org/jira/issues/?jql=project%20%3D%20cassandra%20and%20resolution%20%3D%20unresolved%20and%20labels%20in%20(%27test-fail%27%2C%20%27test-failure%27%2C%20%27testall%27%2C%20%27dtest%27%2C%20%27unit-test%27%2C%20%27unittest%27)%20and%20assignee%20%3D%20null%20order%20by%20created%20ASC We're thankfully still in a place where these tickets are at least being created, but unless there's a body of people that are digging in to fix those test failures they're just going to keep growing. On Fri, Mar 10, 2017 at 5:03 AM, Stefan Podkowinski <s...@apache.org> wrote: > If I remember correctly, the requirement of providing test results along > with each patch was because of tick-tock, where the goal was to have > stable release branches at all times. Without CI for testing each > individual commit on all branches, this just won't work anymore. But > would that really be that bad? Can't we just get away with a single CI > run per branch and day? > > E.g. in the future we could commit to dev branches that are used to run > all tests automatically on Apache CI on daily basis, which is then > exclusively used for that. We don't have that many commits on a single > day, some of them rather trivial, and I think we'd be able to figure out > the one of them causing a regression on the day after. If all tests > pass, we can merge dev manually or even better automatically. If anyone > wants to run tests on his own CI before committing to dev, that's fine > too and will help analyzing any regressions if they happen, as we then > don't have to look at those patches (and all commits before on dev). > > > > On 09.03.2017 19:51, Jason Brown wrote: > > Hey all, > > > > A nice convention we've stumbled into wrt to patches submitted via Jira > is > > to post the results of unit test and dtest runs to the ticket (to show > the > > patch doesn't break things). Many contributors have used the > > DataStax-provided cassci system, but that's not the best long term > > solution. To that end, I'd like to start a conversation about what is the > > best way to proceed going forward, and then add it to the "How to > > contribute" docs. > > > > As an example, should contributors/committers run dtests and unit tests > on > > *some* machine (publicly available or otherwise), and then post those > > results to the ticket? This could be a link to a build system, like what > we > > have with cassci, or just upload the output of the test run(s). > > > > I don't have any fixed notions, and am looking forward to hearing other's > > ideas. > > > > Thanks, > > > > -Jason > > > > p.s. a big thank you to DataStax for providing the cassci system > > >
