Hi all, We’ve had a few threads now about the successes and failures of the tick-tock release process and what to do to replace it, but they all died out without reaching a robust consensus.
In those threads we saw several reasonable options proposed, but from my perspective they all operated in a kind of theoretical fantasy land of testing and development resources. In particular, it takes around a person-week of effort to verify that a release is ready. That is, going through all the test suites, inspecting and re-running failing tests to see if there is a product problem or a flaky test. (I agree that in a perfect world this wouldn’t be necessary because your test ci is always green, but see my previous framing of the perfect world as a fantasy land. It’s also worth noting that this is a common problem for large OSS projects, not necessarily something to beat ourselves up over, but in any case, that's our reality right now.) I submit that any process that assumes a monthly release cadence is not realistic from a resourcing standpoint for this validation. Notably, we have struggled to marshal this for 3.10 for two months now. Therefore, I suggest first that we collectively roll up our sleeves to vet 3.10 as the last tick-tock release. Stick a fork in it, it’s done. No more tick-tock. I further suggest that in place of tick tock we go back to our old model of yearly-ish releases with as-needed bug fix releases on stable branches, probably bi-monthly. This amortizes the release validation problem over a longer development period. And of course we remain free to ramp back up to the more rapid cadence envisioned by the other proposals if we increase our pool of QA effort or we are able to eliminate flakey tests to the point that a long validation process becomes unnecessary. (While a longer dev period could mean a correspondingly more painful test validation process at the end, my experience is that most of the validation cost is “fixed” in the form of flaky tests and thus does not increase proportionally to development time.) Thoughts? -- Jonathan Ellis co-founder, http://www.datastax.com @spyced
