On Tue, Jan 8, 2019 at 11:08 AM Lukas Ruzicka <[email protected]> wrote:
> Hello KDE and XFCE teams, > > I am writing to you on behalf of the Fedora QA team, because you are the > stakeholders of release-blocking desktop environments in Fedora. > > We believe that there is need to revisit and review some of the approaches > we > have in desktop testing. I hope that I can clarify in the text. > > *How do we test desktop applications?* > > Currently, there are three desktop environments, which block the Fedora > releases: GNOME, KDE, and XFCE (on ARM). Among others, there is a test case > called “Desktop Menus” [ > https://fedoraproject.org/wiki/QA:Testcase_desktop_menus ] which is used > for all three desktop environments. > > Why is Xfce tested only on ARM devices? I mean if that's one of the main reasons it's slow and tedious, why not change the architecture? Additionally, it would be probably worthwhile to create two different sets of programs, one group has to start and thus needs to be tested and the other is a nice to have, but doesn't need to start. At least the criteria could be a little bit less strict. Johannes *What is the purpose of the Desktop Menus test case?* > > The purpose of the “Desktop Menus” test case is to test that applications > listed > in the menu do generally work. That means that each application in the > default > system menu must launch without crashing and withstand basic usage. [2] > > *What problems can arise when trying to follow the test case?* > > This approach leaves a lot of space for variability, because the terms are > not > clearly given, especially when talking about “basic functionality” of the > desktop applications. In the past, there have been some disputes during > Blocker > Review meetings, or during Go/No go meetings whether basic functionality is > broken or not. It's usually up to group judgement to decide what is a basic > functionality and what it beyond that, which means as testers we need to > err on > the side of doing more testing rather than less. > > Testing the “Desktop Menus” test case is one of the most time consuming > test > case we currently have. In GNOME, there are currently about 40 > applications in > the default menu and all need to be started and their basic functionality > tested. In KDE, the number of applications is even higher, and some > application > types (like web browsers) are even present as several different > applications. > The scope of this test case also covers testing the subcomponents in the > Control > Center (or a similar equivalent) and, usually, such subcomponents are > numerous. > > Testing XFCE on ARM is also rather time consuming, because the ARM devices > we > can use for testing are very slow and there are some application that make > little sense on an ARM motherboard (e.g. a CD writing software). Thus, > doing a > thorough Desktop Menus test case requires a rather longish time frame and > we > lose time we could devote to different problems. > > Besides that, the test case is also very boring, so it usually gets tested > by > Fedora QA members only, with few inputs from the community. So, before the > Beta > and Final release we are able to do such tests a couple of times, but the > overall coverage is not big. > > *What could we do about it?* > > At Fedora QA, we have been trying to come up with some ideas to make the > situation a little better than it is now. The goal is that the important > areas > get enough attention and testing they deserve, and we avoid shipping > broken and > under-tested software. See the ideas below and please tell us if you could > adopt > all/some of them, or if you have different suggestions to improve the > current > state, we'd love to hear them too. > > > > - The list of pre-installed applications could be pruned of those > which don't make much sense for that particular purpose (e.g. CD writing > software on ARM, or perhaps on any system nowadays). > - Pre-installed applications could be deduplicated where possible > (e.g. not having several web browsers pre-installed by default). > - We could define a list of "important" applications that would be > tested thoroughly and the basic functionality criterion would apply just to > them. That would tell QA where to focus. (This would obviously contain > applications like system settings, file browser, text editor, browser, > terminal emulator, etc. But e.g. a screenshot utility would probably not > fall into the list. Also, in case of duplicated apps, only one of those > could be marked as "important", and the others would* not). The rest of the > applications would be just best effort in terms of QA, and wouldn't block > the release. > - Last but not least, you could help us more with release validation > before relevant milestones (Beta, Final) :-) If there's anything that > should be improved on our part (communication, instructions, etc), we'd > love to hear that. > > Please, let me know what you think about the ideas proposed above. Do you > have > any other ideas that could help us improve the quality (with the same > amount of > testing force)? > > > Best regards, > > -- > > Lukáš Růžička > > FEDORA QE, RHCE > > Red Hat > > <https://www.redhat.com> > > Purkyňova 115 > > 612 45 Brno - Královo Pole > > [email protected] > TRIED AND PERSONALLY TESTED, ERGO TRUSTED. <https://redhat.com/trusted> > _______________________________________________ > xfce mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/[email protected] >
_______________________________________________ xfce mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
