On Fri, Dec 5, 2025 at 6:25 PM Adam Williamson <[email protected]> wrote:
> On Fri, 2025-12-05 at 17:14 +0100, Lukas Ruzicka wrote: > > Hello, > > > > The QA team has been missing test cases to test Basic networking. I have > > written a draft of the test case and now I am collecting suggestions on > how > > to improve it, check if anything is missing, or if something needs to be > > changed. > > > > The Basic networking test case is here: > > https://fedoraproject.org/wiki/QA:Testcase_Basic_networking > > > > We will also need to create a WIFI test case and a VPN test case, both > the > > CLI and GUI variants, so if anybody has ideas how to do it, let me know. > > > > Thanks for your help. > > Thanks for writing this! > > It seems...long. :D I get that it's trying to encapsulate everything in > the criterion, but it's a bit overwhelming. Still, it is very > comprehensive and robust. > > I think when I wrote the criterion I was envisaging something a bit > shorter, sloppier and vaguer, that mostly relied on the tester's > 'typical' network configuration and was just 'try and open a web site, > does that work?' kinda stuff. But...maybe this is better? > > Curious to know what others think. Do we prefer something very detailed > and 'clean room' like this? > Hmmm. First of all, thanks, Lukas, for clearly giving it a lot of effort! Unfortunately, I'm afraid that this amount of effort will make it one of those "nobody wants to do it" test cases, because it's long and complex. But it depends what we want to do with this. If this is to be automated, then big +1 from me, let's do it. If this is intended to be a manual test, then I think we need to simplify it, and not just because of length, that's just one of my concerns. One of my other concerns is that this is... too short. Networking requires a lot of know-how and in many areas it seems it doesn't go deep enough, which means further expansion to explain some steps in more detail would be needed. For example, it uses keywords like "enp1s0" but doesn't explain how to figure out what *your* actual network device is called. The same goes for IP ranges, connection names ("Wired connection 1"), and maybe some more. Using VM as a remote server is of course the most straightforward idea, but it has its own pitfalls. For example, ping doesn't work in libvirt user session, only system session (at least according to my past experience). The IP ranges will vary (unlike in the testcase, mine is 192.168.124.0/24). And of course, the tested system is often a VM itself. Does this testcase assume nested virt in that case? Or VM<->VM connection? This will definitely get even more complex, if we want this to be followed by the general public with heavily varied networking environments. I also see two possible goals of this test case, and I'm not sure which one was the intended one. The first goal is to verify that the very networking basics work, setting an IP address, ping, curl, etc. This test case verifies that. The other goal is to test that the real-world network works on a particular device of the tester. So actually sending packets to your router and to the internet, receiving a web page, etc. Since this test case uses a local VM, are those packets even going through the network card, or are they just "virtualized" in the kernel? I have no idea. But it surely doesn't test that your wifi connection works fine, or cable speed negotiation with your router, and that you can open fedoraproject.org. The first goal is great for automation, the second goal is good for human testers with varied hardware. It might be best if we can do both. Convert this into an OpenQA script that will run tirelessly each compose. The steps are exact, no need for a human tester to repeat it. And let's have a simplified version for humans, that will test connecting to a wifi/cable, pinging a well-known server, opening a website (note that we already have https://fedoraproject.org/wiki/QA:Testcase_desktop_browser for that, though), and possibly some optional stuff like switching a dns server. The wifi seems like the most important stuff to me (for human testing), because we can't easily replicate that in OpenQA (most of the other stuff we can).
-- _______________________________________________ test mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
