On Mon, Nov 28, 2011 at 2:16 PM, Nikos Chantziaras <rea...@arcor.de> wrote: > On 11/28/2011 06:59 PM, Florian Philipp wrote: >> >> Am 28.11.2011 17:15, schrieb Nikos Chantziaras: >>> >>> On 11/28/2011 02:29 PM, Albert W. Hopkins wrote: >>>> >>>> On Sun, 2011-11-27 at 20:28 +0100, Andrea Conti wrote: >>>>> >>>>> With 100% repeatability, mind you, which does raise same questions on >>>>> the amount of testing done before release. Yes, it's ~arch and >>>>> rc_parallel is explicitly marked "experimental", but it's not expected >>>>> to be completely and consistently broken, either. >>>>> >>>>> If that sounds like I'm ranting, it's because I just spent about an >>>>> hour >>>>> getting three machines affected by this problem back into working >>>>> state. >>>>> >>>>> If anyone still has it installed, it's time to sync and downgrade :) >>>> >>>> Sorry to add more to the whining but... >>>> >>>> Yes, you are in the testing tree. Yes, as a member of testing, *you* >>>> expect things will occasionally break, and it is *your* job to test >>>> things, break them, and report bugs. >>> >>> Generally true, but not when something is obviously broken. That means >>> not even its upstream dev bothered to test it. >>> >>> ~arch is for "we think this works, but please give it a go in case there >>> are problems". It's *not* for "we have no idea if this works because we >>> didn't even try it once". >> >> Do you have any idea how much time you can spend with the kind of system >> testing you propose? > > About 2 minutes? Enabling the parallel startup thingy and rebooting the > machine. There you go :-/
That's a facetious answer, and you're purposely only examining a tiny piece of the testing surface. Hindsight is 20/20, though only if you're lucky. Perhaps they've never seen this type of failure before, and they could add a single test to whatever unit test suite they may be using. Perhaps that's an improvement they can make going forward. To fully test OpenRC, you'd want a two-stage testing harness. The outer stage would generate Gentoo VMs with every plausibly-relevant USE flag permutation crossed against as many automatically-generated permutations of OpenRC configuration as could be considered plausibly encountered. For each generated VM, spin it up. Watch for some kind of watchdog "hey, I booted successfully!" indicator. Then spin up a testing harness *inside* the VM to ensure all services started and behave correctly. Dump a report to the vmhost detailing that everything went well (or didn't), and hibernate the VM. vmhost looks at the report and decides whether or not to keep the saved VM state. That's an extraordinary amount of testing to do. And that's what I see argued as what ~arch is for; instead of having a script whip up and test hundreds of virtual machines, people running ~arch do that testing. Gentoo devs get reports for the features and combinations that people actually *use*, and can spend less time fixing features nobody is using. (And it's obvious none of the OpenRC devs are using parallel boot themselves, or they would have caught this. Perhaps that's why it's experimental; nobody who actively uses that feature is keeping up with HEAD and offering patches.) -- :wq
