On 6/4/16, Eric S. Raymond <e...@thyrsus.com> wrote: > We have three major possible pathways to this kind of test coverage. One > is Mozilla rr, the other is some kind of serious simplification attack > on the hairball in the network code, and the third is somebody managing to > grok the hairball in its full, present, hideous glory.
We don't need anything nearly as clever as Mozilla rr or TESTFRAME to achieve the kind of automation I'm asking for. You want to test things at the granularity of system calls (rr) or some abstraction that goes deeper into the code (TESTFRAME). I want to test things at the *user-visible* level. Automate the process of supplying configuration files that exercise a variety of functionality, running them on real hardware and real networks, and monitoring the results with ntpq. That is, take the sort of testing that we're already doing by hand and make it systematic and automatic. > Better collaboration with NTP.org...well, that would sure be nice and > I'd be willing, but I fear pigs will achieve escape velocity before we > get any actual cooperation from them. I won't be more specific on a > list with publicly-accessible archives. > > Do you really think it's realistic to block 1.0 on that? I don't. If it can't be done then it can't be done, but I don't think we've exhausted our options. I'll follow up off-list. > I have a little list, I have a little list... > > Unfortunately you're entering another political minefield here. At > least two features I want to remove would cause a political shitstorm > if word got out that we were thinking of diking them out *before* I > have solid white papers to show they're nugatory. Okay, if some of the featurectomies are going to be controversial no matter what, then we don't have to block 1.0 on those. But let's do the rest of them so that we're not (justifiably) accused of gratuitously breaking people's setups post-1.0. > I agree with the intent to revisit using something Phabricator-like. > I disagree that that should be coupled with 1.0. Stability ought > to be measured by er, *stability*, not by whether we've gone through > that process. We're using two different senses of "stable". You're using it to mean "free of crashes". I'm using it way Debian uses it: a version is stable when it ceases to require frequent patching, and if a particular, serious issue comes up (usually but not always security-related) then the maintainers will provide an update that fixes that issue and leaves everything else untouched. A corollary to something being stable in this sense is that we can afford high overhead for making changes, because we don't have to make very many of them. _______________________________________________ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
