On Thu, 2017-11-09 at 20:37 -0500, Sumantro Mukherjee wrote:
> Hey All,
>
> Now that we have the F27 Modular Server Beta, It will be great to have the
> test cases.
> I've started writing them, I would like to have the feedback and thoughts on
> them.
> I have mostly used dnf module --help as a base to derive the these. If
> something is
> missing, I would love to draft them too, just let me know.
>
> [1] Listing all modules:
> https://fedoraproject.org/wiki/User:Sumantrom/Draft/FMS_module_list
> [2] Enabling & Disabling modules:
> https://fedoraproject.org/wiki/User:Sumantrom/Draft/FMS_enable_disable_module
> [3] Install & Update :
> https://fedoraproject.org/wiki/User:Sumantrom/Draft/dnf_install_remove_update_module
> [4] Module Info :
> https://fedoraproject.org/wiki/User:Sumantrom/Draft/dnf_module_info
Along with the suggestions from others, I'd say:
1) Make the setup steps more generic. It is a bad idea to talk about
'Fedora 27' and 'modular server' and even worse to include download
links. Think about the philosophy behind the whole release validation
design in the wiki: the test cases are made as generic ("vague") as
possible. All the context about running the test for a specific
release, or image, or whatever, is provided at a different level: a
validation event page which specifies the compose it's for and provides
download links, for instance, or a Test Day, which similarly specifies
what particular image etc. is being tested at that Test Day. We keep
that information in the validation event page or the Test Day page, not
in the test case, so we don't have to continually update test case
pages and duplicate them when we want to run the same test in a
different context.
Take a look at the steps in existing post-install validation test
cases, for e.g.
https://fedoraproject.org/wiki/QA:Testcase_base_update_cli . The text
there is a pretty good guide:
"Clean boot the Fedora you wish to test: this could be a system
installed from a particular snapshot, pre-release, or release, or a
live image. It should be an image for which updates will be available
(or you can downgrade a package after installation)."
I'd say to use something like that, but just specify that it must be a
*modular* installation.
2) Similarly, the test case names should *not* include 'FMS' or any
other reference to modular server specifically; in future, more things
than just Server are going to use the Modularity stuff. I'd suggest
consistent names something like dnf_module_(something). Perhaps:
dnf_module_list
dnf_module_enable_disable
dnf_module_install_remove_update
dnf_module_info
2) Remove the 'optional' sections - what you've left in is just sample
text from the template, don't leave it in if there aren't actually any
optional steps. (Also you seem to have some sort of template syntax
error in User:Sumantrom/Draft/dnf_module_info ).
3) FMS_enable_disable_module should probably include some kind of
verification as to whether the enablement/disablement actually worked.
4) I'm not sure I like the capitalization of Module, Stream, Install
Profile etc in dnf_install_remove_update_module . I'd probably put them
in italics instead - ''module'', ''stream'', etc.
5) Similarly to 3), the install_remove_update test should include
verification that the installation / removal / update was actually
performed.
6) It might be worth separating install/remove/update into two tests,
one for update, one for install/remove.
Thanks for wokring on this!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
