On Dec 17, 2013, at 1:07 AM, Adam Williamson <awill...@redhat.com> wrote:
> On Tue, 2013-12-17 at 00:57 -0700, Chris Murphy wrote: >> On Dec 16, 2013, at 6:33 PM, Adam Williamson <awill...@redhat.com> wrote: >> >>> Actually - it'd basically just be the 'guided installation' table from >>> https://fedoraproject.org/wiki/User:Adamwill/Draft_storage_matrix#Guided_installationwithout >>> all the other bits, basically. >> >> I basically get this. I expect there'll be separate matrices or pages for >> the archs? > > oh, god, handwave handwave. > > That's the 'matrix needs to exist in three dimensions' problem: there's > just too many damn variables. What do we do if for a given test it > matters what storage format you use, *and* whether you're UEFI or BIOS, > *and* whether you, I dunno, pick encryption or not? The fundamental > problem that makes storage validation design such a hard problem to > solve is there's just too many inter-related factors. If you draw up the > table to test every operation under all the different possible > conditions in which you could test it, it's just unmanageable. And we're just talking guided partitioning right now, and yet we need to put it in a hecatonicosachoron to visualize the ensuing matrix. But yeah other than ostrich maneuver what's the alternative? The storage layouts are in fact different between x86_64 and EFI as are the bootloaders. The least difference is i386 and x86_64, but then there could be a unique bug (?) that only affects one arch? I dunno that's far fetched but I suppose it could happen. > On a meta level, though, I think it's not > possible for us to come up with "the perfect" storage validation test > plan and have it be actually viable to accomplish: we're going to have > to cut out _some_ variables. It's just a question of which combinations > are the most important and practical to test, and which we have to pass > on. I'm not looking for perfect. I'm just trying to get an idea of what there really is to test. I haven't gotten to triaging it yet. I think we need to see the liabilities, show which ones we'll test, and the rest of of that f'n hecatonicosachoron's cells are holes for someone else to decide the consequences. As in, get more QA people. Stop the feature creep. Placard Manual Partitioning as "best effort", or spun out into a separate project. Manual Partitioning is highly vertical use case, OS install only, it doesn't help people create storage for general purpose scenarios. Yet it could. And further vertical use case, is that it's Fedora/RHEL specific. So I just don't see how the wider community really contributes to that in terms of either code or testing. Difficult to imagine. >> But I'd guess the QA:Testcase_dualboot_with_windows tests for >> existing GPT, and EFI System partition, rather than actually creating >> them (correctly). I'll bet the code that does that is called once for >> all of the guided partition schemes. So it seems like if we check the >> Windows case to make sure it doesn't break Windows, we also get a "use >> existing GPT and ESP" test of code; but still need one that "creates >> new GPT and ESP" layout test of the code. Yes? I did that on Mac EFI, >> but it's got slightly different code. > > Well, define 'need' :/ Need defined by an answer of no to the question, do we want to ship a release where an install fails (including fails to boot) on EFI because an ESP wasn't created for some reason? Another way to do this is have the full matrix of possibilities but so long as each arch has *one* guided partitioning method that's known to work, we are a go even if the other possibilities aren't tested. If it turns out that one option was LVM thinp and everything else was busted, well so be it. More than likely we're going to have alarm bells going off in the community during beta if the basic paths are busted, so I don't know that we need to exercise those that much. > It's the same problem as above - too many damn > paths, too much stuff that we would _like_ to cover in an ideal world. > We have to somehow make a determination about what we don't cover, > because we can't cover everything. One of the LVM options in guided needs to go. Once LVM thinp is mature enough, it's better for all use cases. But until then there should be one LVM option in guided partitioning. > >> >>> Do we think it's worth taking that bit out of the larger storage rework >>> proposal and sticking it in the current matrix, replacing the >>> appropriate existing test cases? It would be an hour or two's work for >>> me, I guess. Eh, I guess I'll just draft it up and send it out and see >>> what you all think. >> >> Why not just delete the appropriate existing test cases rather than merging >> them? > > That was what I meant by 'replacing the appropriate existing test cases' > - if we like this plan we put the new matrix in and take the old test > cases out. I read yours has moving bits from the new larger matrix to the existing matrix, taking about 2 hours of work. I'm suggesting just keep the new larger matrix as is, and delete the corresponding tests in the old matrix. Deleting takes less time than merging Chris Murphy -- test mailing list test@lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
