Re: Criteria proposal: applying 'post-install' criteria to live and appliance images
2016-01-29 20:44 GMT+02:00 Adam Williamson : > On Fri, 2016-01-29 at 08:46 -0500, Kamil Paral wrote: > > > I'm not completely happy about the wording of: > > " This criterion does apply to live environments. However, a stricter > > standard of judgement may be applied to conditional violations in > > live environments, as clean shutdown and log out functionality is > > relatively less important on a live boot than an installed system. " > > https://fedoraproject.org/wiki/User:Adamwill/Draft_Beta_Criteria_Post > > install#Shutdown.2C_reboot.2C_logout > > > > For example logout is necessary if you want to switch languages (and > > our l10n test days rely on that). Reboot and shutdown is necessary > > for automating stuff. I'd use the same measure as in post-install > > here. > > Hmm, IIRC this was one case that *really happened*, and I was trying to > catch the flavor of our IRC discussion at the time - my memory is that > we were willing to accept such bugs as blockers, but we'd maybe be more > likely to waive them for only affecting a small amount of users or > being workaroundable or something like that. I can go back and check > the logs again, though. What do other folks think? > The Gnome logout hang/delay bug in Fedora 23 prevented changing the language in live images for me: I asked Adam about proposing it for a blogger. I never proposed it as a blocker as the issue got accepted under more straightforward criteria soon after. I would rather have an explicit criteria for language change for beta if there is need to have working mechanism to change language before final release. -- Kari Koskinen -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: installing via VNC over IPv6.
Firefox, to find Cockpit running on Fedora server wants the IPv6 address in the form: http://[2601:282:702:b65c:a871:6172:b20f:3a0b]:9090 And then ssh wants it in the form ssh chris@2601:282:702:b65c:a871:6172:b20f:3a0b But I don't actually know what form TigerVNC wants it in. I just tried nmap -6 -A -T4 and it finds this # nmap -6 -A -T4 2601:282:702:b65c:baae:edff:fe77:ea51 Starting Nmap 7.00 ( https://nmap.org ) at 2016-01-29 16:22 MST Nmap scan report for 2601:282:702:xx Host is up (0.00089s latency). Not shown: 999 closed ports PORT STATE SERVICE VERSION 5901/tcp open vnc VNC (protocol 3.8) | vnc-info: | Protocol version: 3.8 | Security types: | None (1) |_ WARNING: Server does not require authentication OK so that seems like it's listening on port 5901 via IPv6. But what to plug into TigerVNC? DOH! [IP]:1 That works. OK I just didn't iterate enough. And once again it's user error. Chris Murphy -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: installing via VNC over IPv6.
On Fri, Jan 29, 2016 at 3:39 PM, Ian Pilcher wrote: > On 01/29/2016 03:42 PM, Chris Murphy wrote: >> >> The problem I'm running into next is that non-lives don't have either >> netstat or ss, so I can see if xvnc is actually listening on an ipv6 >> port or not. > > > Can you install using some other method? > > If so ... do the installation, boot the installation media with the > VNC & IPv6 options, and mount the installed system as a chroot. Then > you can use any tools in the installed system. Neither netstat nor ss show even the active ipv4 connection. Even if I connect via VNC with that ipv4 address, nothing is listed from within the chroot using netstat. Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State Active UNIX domain sockets (w/o servers) Proto RefCnt Flags Type State I-Node Path Active Bluetooth connections (w/o servers) Proto Destination SourceState PSM DCID SCID IMTUOMTU Security Proto Destination SourceState Channel -- Chris Murphy -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: installing via VNC over IPv6.
On Fri, Jan 29, 2016 at 3:39 PM, Ian Pilcher wrote: > On 01/29/2016 03:42 PM, Chris Murphy wrote: >> >> The problem I'm running into next is that non-lives don't have either >> netstat or ss, so I can see if xvnc is actually listening on an ipv6 >> port or not. > > > Can you install using some other method? > > If so ... do the installation, boot the installation media with the > VNC & IPv6 options, and mount the installed system as a chroot. Then > you can use any tools in the installed system. Neat. I'll try that. -- Chris Murphy -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: installing via VNC over IPv6.
On 01/29/2016 03:42 PM, Chris Murphy wrote: The problem I'm running into next is that non-lives don't have either netstat or ss, so I can see if xvnc is actually listening on an ipv6 port or not. Can you install using some other method? If so ... do the installation, boot the installation media with the VNC & IPv6 options, and mount the installed system as a chroot. Then you can use any tools in the installed system. -- Ian Pilcher arequip...@gmail.com "I grew up before Mark Zuckerberg invented friendship" -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
installing via VNC over IPv6.
Hi, I posted a variation of this question on users@ and it looks like no one there has hit this. Basically I can ssh, and reach Cockpit, via IPv6. But I can't reach Anaconda over VNC via IPv6. This says it ought to work. https://fedoraproject.org/wiki/Anaconda/Features/Ipv6OnlyInstallation The problem I'm running into next is that non-lives don't have either netstat or ss, so I can see if xvnc is actually listening on an ipv6 port or not. So my hammer approach is to ask if it's possible to get a Rawhide boot.iso with ss or netstat included? :-P But really, I'm asking what's the next troubleshooting step given that netstat and ss aren't available? Asking on anaconda@ or filing a bug seems premature, but maybe that's wrong seeing as it should just work but isn't? -- Chris Murphy -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Fedora 24: i686 images no longer 'release blocking'
On Fri, 29 Jan 2016 10:49:06 -0800 Adam Williamson wrote: > i.e. they're actually marked 'i386' and are in a directory called > 'i386'. > > CCing Jan and nirik (who's shown as creating the page in the edit > log). Yeah, just missed/typo/mistake. I have changed them to the i386 path and marked them "no" for blocking. kevin pgpUsHtt99rkj.pgp Description: OpenPGP digital signature -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: In A World Where...TCs don't exist?
On Fri, Jan 29, 2016 at 10:07:14AM -0600, Dennis Gilmore wrote: > > > > https://fedorahosted.org/rel-eng/ticket/5805 [...] > We have none of that info and its not yet on our radar. > https://fedoraproject.org/wiki/ReleaseEngineering/PriorityPipeline is > the list of things we have coming up. I think there are some not > documented as well as they should be. I think this request just kind of fell through and didn't get included in the new prioritization process. Let's get it added in as something for _sometime_ in the future, even if it doesn't bump current priorities. -- Matthew Miller Fedora Project Leader -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Fedora 24: i686 images no longer 'release blocking'
On Fri, 2016-01-29 at 08:55 -0500, Kamil Paral wrote: > > I have tweaked the release criteria 'preamble' text slightly to mention > > this explicitly, and also to link to the canonical list of release- > > blocking images that the program manager is maintaining now (the F24 > > list is > > https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24 > > ). > > I wonder why these two images are marked as release blocking? > > Cloud_Images/x86_64/Images/Fedora-Cloud-Base-_RELEASE_MILESTONE_-_DATE_.i686.qcow2 > > Cloud_Images/x86_64/Images/Fedora-Cloud-Base-_RELEASE_MILESTONE_-_DATE_.i686.raw.xz Well, it might be because they're listed as being inside the /x86_64/ directory, and whoever's job it was to put 'no' by all the i686 images saw that but missed the 'i686' in the image name. I think the listed names and locations are wrong; at least according to our (fedfind-generated) download table for 23 Final RC10, the locations were: https://dl.fedoraproject.org/pub/alt/stage/23_RC10/Cloud_Images/i386/Images/Fedora-Cloud-Base-23-20151030.i386.raw.xz https://dl.fedoraproject.org/pub/alt/stage/23_RC10/Cloud_Images/i386/Images/Fedora-Cloud-Base-23-20151030.i386.qcow2 i.e. they're actually marked 'i386' and are in a directory called 'i386'. CCing Jan and nirik (who's shown as creating the page in the edit log). -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Criteria proposal: applying 'post-install' criteria to live and appliance images
On Fri, 2016-01-29 at 08:46 -0500, Kamil Paral wrote: > > Since people seemed to be on board with this approach, I've done the > > drafts. Here they are: > > > > https://fedoraproject.org/wiki/User:Adamwill/Draft_Alpha_Criteria_Postinstall > > https://fedoraproject.org/wiki/User:Adamwill/Draft_Beta_Criteria_Postinstall > > https://fedoraproject.org/wiki/User:Adamwill/Draft_Final_Criteria_Postinstall > > > > You can see the diff to the current criteria in the page history (I > > first saved an exact copy of the current criteria, then made the > > changes). Any thoughts on the specific changes are welcome! Thanks. > > Thanks for the drafts. > > I hesitate whether we should mandate working updates in live environment: > https://fedoraproject.org/wiki/User:Adamwill/Draft_Alpha_Criteria_Postinstall#Updates Er, we wouldn't? That criterion still explicitly states "The installed system". When a criterion had text like that I didn't think it necessary to *also* add some text saying "doesn't apply to live environments", since that's just two ways of saying the same thing. > Installing anything else than just a few small packages usually > doesn't work at all, because you run out of allocated disk overlay > memory (which seems to happen very soon even for systems with large > amounts of physical RAM). The question is whether we want this to > work "in a reasonable degree" (small updates), or whether we don't > want to require it on Live at all. Yes, it was entirely my intent that this one not apply to live systems. > I'm not completely happy about the wording of: > " This criterion does apply to live environments. However, a stricter > standard of judgement may be applied to conditional violations in > live environments, as clean shutdown and log out functionality is > relatively less important on a live boot than an installed system. " > https://fedoraproject.org/wiki/User:Adamwill/Draft_Beta_Criteria_Post > install#Shutdown.2C_reboot.2C_logout > > For example logout is necessary if you want to switch languages (and > our l10n test days rely on that). Reboot and shutdown is necessary > for automating stuff. I'd use the same measure as in post-install > here. Hmm, IIRC this was one case that *really happened*, and I was trying to catch the flavor of our IRC discussion at the time - my memory is that we were willing to accept such bugs as blockers, but we'd maybe be more likely to waive them for only affecting a small amount of users or being workaroundable or something like that. I can go back and check the logs again, though. What do other folks think? Thanks for checking the drafts! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Fedora 24: i686 images no longer 'release blocking'
On Fri, 2016-01-29 at 08:09 -0500, Kamil Paral wrote: > But if we want to still test i686 in OpenQA (at least on a best- > effort basis), we somewhat rely on the wiki pages for reporting. (Or > do you think that having it in OpenQA frontend is good enough?). Well, there's one other venue where we get the info: the 'compose check' reports. I'm honestly rather happy with those. They do the job; I noticed as soon as i686 broke the other day. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: In A World Where...TCs don't exist?
On Fri, 2016-01-29 at 09:26 -0500, Kamil Paral wrote: > Here's a question. Are we going to "nominate" only those composes in > which a substantial component changed (i.e. anaconda or systemd), > similarly to what we do now in rawhide, or are we going to nominate > each new compose (i.e. one or more per day)? That's definitely something to consider, yeah. It's logic that's quite easy to tweak. > The first approach seems simpler for humans, but I can't imagine how > we make it work for e.g. Desktop matrices - there's so many > components in there that we would probably end up nominating every > day anyway. Well, I intentionally never tried to extend the list of 'significant packages' to every single one which could *possibly* cause anaconda's behaviour to change, and I wouldn't suggest it would make sense to do that for GNOME either. Really it just seemed like a neat way of regulating the flow of nominated composes. Note the mechanism is a bit more complex than you mentioned, there are a pair of time constraints: it *always* waits at least three days between nominations, and if two weeks go by without a 'significant' package change it'll go ahead and nominate anyway (that may have kicked in once :>). > The second approach means we would let automation do its job and > humans would have to rely mainly on testcase_stats to see which test > cases were recently tested and which were not, and test according to > that. I think the second approach is something that we should aim for > in the future, but I'm not sure we're there yet. It will certainly > require some larger changes in testcase_stats to make sure they > correctly represent everything (now that we'll rely solely on that), > e.g. not squashing different test environments together into a single > result, etc. This is broadly my take, yeah. Honestly, I think it might be time to go back into the test framework jungle, though we might actually wind up in the dreaded 'build our own' position this time. I've been vaguely thinking about a system to consolidate automated and manual test results into resultsdb. So we'd have something that would submit results from autocloud and openQA to resultsdb, and we'd build some kind of client (webapp or whatever) for submission of manual test results, and displaying all the combined results from automated test systems and manual testers. In my mind this system doesn't actually store or display test cases; they stay in the wiki. Each test case has a permanent ID and a changeable URL, so we can rename test cases where appropriate. The new bits would simply link out to the wiki where appropriate. It's still just a concept for now, but that's kinda where my mind's going...WDYT? Do you see more mileage in extending testcase_stats? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Fedora Rawhide 20160129 compose check report
Missing expected images: Kde disk raw armhfp Cloud_atomic disk raw x86_64 Images in this compose but not Rawhide 20160128: Cloud disk raw i386 Games live x86_64 Design_suite live x86_64 Generic boot x86_64 Cloud disk qcow x86_64 Soas live x86_64 Xfce live x86_64 Cloud vagrant libvirt x86_64 Cloud vagrant virtualbox x86_64 Mate live x86_64 Security live x86_64 Scientific_kde live x86_64 Lxde live x86_64 Workstation live x86_64 Cloud disk raw x86_64 Cinnamon live x86_64 Cloud disk qcow i386 Kde live x86_64 No images in Rawhide 20160128 but not this. Failed openQA tests: 17 of 69 ID: 4579Test: i386 workstation_live default_install URL: https://openqa.fedoraproject.org/tests/4579 ID: 4575Test: x86_64 kde_live default_install@uefi URL: https://openqa.fedoraproject.org/tests/4575 ID: 4574Test: x86_64 kde_live default_install URL: https://openqa.fedoraproject.org/tests/4574 ID: 4567Test: i386 generic_boot default_install URL: https://openqa.fedoraproject.org/tests/4567 ID: 4573Test: i386 kde_live default_install URL: https://openqa.fedoraproject.org/tests/4573 ID: 4553Test: i386 universal server_scsi_updates_img URL: https://openqa.fedoraproject.org/tests/4553 ID: 4552Test: i386 universal server_repository_http_graphical URL: https://openqa.fedoraproject.org/tests/4552 ID: 4551Test: i386 universal package_set_minimal URL: https://openqa.fedoraproject.org/tests/4551 ID: 4554Test: i386 universal server_simple_encrypted URL: https://openqa.fedoraproject.org/tests/4554 ID: 4555Test: i386 universal server_software_raid URL: https://openqa.fedoraproject.org/tests/4555 ID: 4556Test: i386 universal server_btrfs URL: https://openqa.fedoraproject.org/tests/4556 ID: 4557Test: i386 universal server_ext3 URL: https://openqa.fedoraproject.org/tests/4557 ID: 4558Test: i386 universal server_lvmthin URL: https://openqa.fedoraproject.org/tests/4558 ID: 4561Test: i386 universal package_set_kde URL: https://openqa.fedoraproject.org/tests/4561 ID: 4511Test: x86_64 universal package_set_kde URL: https://openqa.fedoraproject.org/tests/4511 ID: 4559Test: i386 universal upgrade_desktop_32bit URL: https://openqa.fedoraproject.org/tests/4559 ID: 4560Test: i386 universal upgrade_2_desktop_32bit URL: https://openqa.fedoraproject.org/tests/4560 Passed openQA tests: 49 of 69 3 openQA tests may be still running or broken! -- Mail generated by check-compose: https://git.fedorahosted.org/cgit/fedora-qa.git/tree/check-compose -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: In A World Where...TCs don't exist?
On Friday, January 29, 2016 02:25:49 PM Richard W.M. Jones wrote: > On Fri, Jan 29, 2016 at 03:03:33PM +0100, Stanislav Ochotnicky wrote: > > On Fri 29 Jan 2016 02:51:31 PM CET Richard W.M. Jones wrote: > > > On Fri, Jan 29, 2016 at 04:33:19AM -0800, Adam Williamson wrote: > > >> Hi, folks! I thought this might be about the appropriate time to throw > > >> this out there. > > >> > > >> There hasn't been a big news press on this, but some of you may know > > >> that releng is fairly close to switching over to Pungi 4 for composes. > > >> For those of you who don't know: > > >> > > >> releng is fairly close to switching over to Pungi 4 for composes. > > > > > > [...] > > > > > > Any chance you can publish metadata for these releases? ie. this 2 > > > > > > year old request: > > > https://fedorahosted.org/rel-eng/ticket/5805 > > > > > > We're in the awkward situation now where OpenSUSE and Ubuntu publish > > > machine-readable metadata, but Fedora does not (or if it now does, > > > please point me to it so we can start using it). > > > > > > Many people would test the cloud images and test their software on > > > > > > cloud images if they could do: > > > $ virt-builder fedora-rawhide > > > $ virt-builder fedora-nightly-MMDD > > > > > > or whatever to get them. > > > > I think you might be looking for something like this? > > https://kojipkgs.fedoraproject.org/compose/rawhide/latest-Fedora-/compose/ > > metadata/ > > > > See the files in the directory for details, be aware the rpm one is huge > > though :-) > > Possibly. > > Really we're looking for cloud images though (ie. *.qcow2), not > install ISOs or trees. I thought Pungi did both? > > There are a few missing fields we require too: > > - size of the disk image (especially when the image xz-compressed, we >need the uncompressed size in order to plan how to resize it) > > - format of the disk image > > - name of the root filesystem (so we can resize the image intelligently) > > - cryptographically-secure checksum of the image > > - libosinfo database key (so we know what emulated devices to present) > > And the metadata should be GPG signed. > > I've got an example here: > > http://libguestfs.org/download/builder/index.asc > > I'm not hung up on the specific format -- for Ubuntu they use a thing > called "SimpleStreams" which we implemented support for -- but it > needs to contain the same or a subset of that metadata. > > Rich. We have none of that info and its not yet on our radar. https://fedoraproject.org/wiki/ReleaseEngineering/PriorityPipeline is the list of things we have coming up. I think there are some not documented as well as they should be. Dennis signature.asc Description: This is a digitally signed message part. -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: In A World Where...TCs don't exist?
> If we have automated, more-than-nightly composes that look much like a > regular release compose would, there's no clear case for having TCs at > all. We could simply stop building them and extend the "nightly" > validation process. I think the way to do that would be to keep > 'nominating' nightly composes for validation testing all the time, > *except* when we're doing RCs. So instead of going something like: > > 24 Rawhide 20160120 > 24 Rawhide 20160215 > == BRANCH POINT == > 24 Branched 20160301 > 24 Branched 20160315 > 24 Alpha TC1 > 24 Alpha TC2 > == ALPHA FREEZE == > 24 Alpha RC1 > 24 Alpha RC2 > == ALPHA RELEASE == > 24 Beta TC1 > > > we'd go something like: > > 24 Rawhide 20160120 > 24 Rawhide 20160215 > == BRANCH POINT == > 24 > Branched 20160301 > 24 Branched 20160315 > 24 Branched 20160401 > 24 Alpha RC1 > 24 > Alpha RC2 > == ALPHA RELEASE == > 24 Branched 20160501 > 24 Branched > 20160515 > 24 Beta RC1 > Here's a question. Are we going to "nominate" only those composes in which a substantial component changed (i.e. anaconda or systemd), similarly to what we do now in rawhide, or are we going to nominate each new compose (i.e. one or more per day)? The first approach seems simpler for humans, but I can't imagine how we make it work for e.g. Desktop matrices - there's so many components in there that we would probably end up nominating every day anyway. The second approach means we would let automation do its job and humans would have to rely mainly on testcase_stats to see which test cases were recently tested and which were not, and test according to that. I think the second approach is something that we should aim for in the future, but I'm not sure we're there yet. It will certainly require some larger changes in testcase_stats to make sure they correctly represent everything (now that we'll rely solely on that), e.g. not squashing different test environments together into a single result, etc. -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Fedora 24: i686 images no longer 'release blocking'
> I have tweaked the release criteria 'preamble' text slightly to mention > this explicitly, and also to link to the canonical list of release- > blocking images that the program manager is maintaining now (the F24 > list is > https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24 > ). I wonder why these two images are marked as release blocking? Cloud_Images/x86_64/Images/Fedora-Cloud-Base-_RELEASE_MILESTONE_-_DATE_.i686.qcow2 Cloud_Images/x86_64/Images/Fedora-Cloud-Base-_RELEASE_MILESTONE_-_DATE_.i686.raw.xz -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Criteria proposal: applying 'post-install' criteria to live and appliance images
> Since people seemed to be on board with this approach, I've done the > drafts. Here they are: > > https://fedoraproject.org/wiki/User:Adamwill/Draft_Alpha_Criteria_Postinstall > https://fedoraproject.org/wiki/User:Adamwill/Draft_Beta_Criteria_Postinstall > https://fedoraproject.org/wiki/User:Adamwill/Draft_Final_Criteria_Postinstall > > You can see the diff to the current criteria in the page history (I > first saved an exact copy of the current criteria, then made the > changes). Any thoughts on the specific changes are welcome! Thanks. Thanks for the drafts. I hesitate whether we should mandate working updates in live environment: https://fedoraproject.org/wiki/User:Adamwill/Draft_Alpha_Criteria_Postinstall#Updates Installing anything else than just a few small packages usually doesn't work at all, because you run out of allocated disk overlay memory (which seems to happen very soon even for systems with large amounts of physical RAM). The question is whether we want this to work "in a reasonable degree" (small updates), or whether we don't want to require it on Live at all. I'm not completely happy about the wording of: " This criterion does apply to live environments. However, a stricter standard of judgement may be applied to conditional violations in live environments, as clean shutdown and log out functionality is relatively less important on a live boot than an installed system. " https://fedoraproject.org/wiki/User:Adamwill/Draft_Beta_Criteria_Postinstall#Shutdown.2C_reboot.2C_logout For example logout is necessary if you want to switch languages (and our l10n test days rely on that). Reboot and shutdown is necessary for automating stuff. I'd use the same measure as in post-install here. -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
Re: Fedora 24: i686 images no longer 'release blocking'
> https://fedoraproject.org/wiki/Template:Installation_test_matrix > > quite a lot of the tables have 'i386' and 'x86_64' as environments. > Especially with the Milestone column, listing i386 alongside x86_64 is > a bit misleading if i386 is no longer blocking. I can see a few > options: > > 1) just ditch the i386 columns entirely; openQA can continue testing > it, and people can test manually if they want, but we don't bother > tracking the results in the validation pages > > 2) stick an admon template at the top of the page saying 'the i386 > tests aren't blocking', with links out to the criteria and/or the FESCo > ticket > > 3) Duplicate each table which distinguishes between 'i386' and > 'x86_64', so we have one table with just an 'i386' column and all tests > marked Optional, and another table with the other columns and the > appropriate milestone I think the answer here is largely dependent on what we want to do about OpenQA. If we didn't have OpenQA at all, I'd do #1, and I'd also duplicate Workstation* and Server* rows in "Default boot and install" section, gray out x86_64 and UEFI columns, and mark the rows as optional. Therefore i686 would be handled the same way we handle spins - there's a way to mark a critical error which prevents default install and boot in the matrix, but that's it. The same solution would probably apply if we decided to drop i686 testing from OpenQA. But if we want to still test i686 in OpenQA (at least on a best-effort basis), we somewhat rely on the wiki pages for reporting. (Or do you think that having it in OpenQA frontend is good enough?). So if we want to keep the matrices around for that purpose, I'd either do #3 and collapse them by default, or I'd create a separate wiki page just for i686 and direct OpenQA results there. This way we can still easily see what was and what wasn't tested, and tools like testcase_stats work for it, but we the core wiki matrices are not overflowing with non-essential stuff. But it is some work and maintenance, and I'm not sure it's worth it. Maybe the OpenQA frontend is just good enough? -- test mailing list test@lists.fedoraproject.org To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org
In A World Where...TCs don't exist?
Hi, folks! I thought this might be about the appropriate time to throw this out there. There hasn't been a big news press on this, but some of you may know that releng is fairly close to switching over to Pungi 4 for composes. For those of you who don't know: releng is fairly close to switching over to Pungi 4 for composes. This will have various interesting effects on QA and the whole process of building Fedora releases. With the current releng process, TC / RC composes are one beast, and nightly composes are another, very different beast. In fact nightly composes barely really 'exist' at all - when we say 'nightly compose' we really mean 'pungify the rawhide/branched repo, and fire off a bunch of koji tasks'. After the fact, there is no real relationship between any of those bits, which is why I had to write fedfind to go out and synthesize the concept of a 'nightly compose' by finding all the Koji tasks and treating them plus the repository boot.iso's as a single 'compose'. With Pungi 4, all composes will look a lot more similar. 'nightly' composes (which, in point of fact, will probably happen more than once per day - I'm not sure if we came up with a new name yet) look a lot more like current TC/RC composes than current nightly composes. You can see approximately what a Pungi 4 compose currently looks like here: https://kojipkgs.fedoraproject.org/compose/rawhide/ as of right now, the Koji built bits - lives, cloud and ARM disk images, etc - aren't integrated with the installer images, but they *will* be, and they'll all show up in the same location. As you can see it has all the different variants, and a Server DVD image. (A Pungi 4 compose also has a bunch of metadata, which means we can more or less kill off fedfind, thank God). The implication of this I wanted to talk about in this thread is: what does this mean for the release validation process, in terms of what composes we cut and what release validation events we have? So as you probably know, right now, the validation process is built around the milestone 'TC' and 'RC' images. We build a series of Alpha TCs and run a bunch of tests for each of these composes, reporting the results to wiki pages named for the composes. Then we do Alpha RCs, then Beta TCs, and so on through Final RCs. For the last few releases we've added on some 'nightly' validation events, where we create wiki pages named for nightly composes and run the same set of tests on the nightly boot.iso's and Koji images, but these have been framed as kind of an 'early warning system' for use before Alpha TC1 arrives, and once Alpha TC1 arrives we stop doing the nightly validation events. With Pungi 4, I don't think this makes a lot of sense any more. Dennis and I have been talking about this and I think we broadly agree on it. TCs and RCs used to be kinda the only way we *could* do validation testing. For long periods we didn't have reliable nightly builds of Rawhide or Branched at all, certainly not all the Koji-produced images. The process for doing 'real' composes was quite long and painful and required squishy human intervention. If we have automated, more-than-nightly composes that look much like a regular release compose would, there's no clear case for having TCs at all. We could simply stop building them and extend the "nightly" validation process. I think the way to do that would be to keep 'nominating' nightly composes for validation testing all the time, *except* when we're doing RCs. So instead of going something like: 24 Rawhide 20160120 24 Rawhide 20160215 == BRANCH POINT == 24 Branched 20160301 24 Branched 20160315 24 Alpha TC1 24 Alpha TC2 == ALPHA FREEZE == 24 Alpha RC1 24 Alpha RC2 == ALPHA RELEASE == 24 Beta TC1 we'd go something like: 24 Rawhide 20160120 24 Rawhide 20160215 == BRANCH POINT == 24 Branched 20160301 24 Branched 20160315 24 Branched 20160401 24 Alpha RC1 24 Alpha RC2 == ALPHA RELEASE == 24 Branched 20160501 24 Branched 20160515 24 Beta RC1 note: all dates completely made up, this is just for illustration. I think it would be plausible to do this for Fedora 24, if the Pungi 4 switchover happens soon and goes well. There would be some details to pin down in relval and wikitcms and stuff (we might need to tweak the validation event naming approach a bit so that it's possible to identify the sequence of events from the names - i.e. so you know where the RCs fit in), but nothing unsolvable. We'll be talking about a lot of this stuff at DevConf, if anyone's going to be there, pin down me or Dennis or someone else involved in release-y stuff and we'd be happy to discuss it. But I wanted to throw something up on the lists for discussion as well. What do you think? Thanks! One point that's come up already is the way that we manually pull newer packages to fix blocker/FE bugs into TC and RC composes via the 'bleed' repo. We're currently envisaging something like the 'buildroot override
