On 12/6/25 09:22, Alyssa Ross wrote: > Demi Marie Obenour <[email protected]> writes: > >> On 12/6/25 08:36, Alyssa Ross wrote: >>> Demi Marie Obenour <[email protected]> writes: >>> >>>> On 12/6/25 07:32, Alyssa Ross wrote: >>>>> Demi Marie Obenour <[email protected]> writes: >>>>> >>>>>> On 12/6/25 07:26, Alyssa Ross wrote: >>>>>>> Demi Marie Obenour <[email protected]> writes: >>>>>>> >>>>>>>> While trying to sandbox the file chooser portal, I broke it. >>>>>>>> This caused files not to be saved, resulting in silent data loss. >>>>>>>> Unfortunately, the integration test still passed. >>>>>>>> >>>>>>>> Is this a bug in the test? Is there a better alternative to manual >>>>>>>> testing? >>>>>>> >>>>>>> Not presently, but we can work on improving the test. The current >>>>>>> portal test was written as a regression test for a specific issue we >>>>>>> had. It's quite hard to test completely end to end but we could do a >>>>>>> lot better. >>>>>>> >>>>>>> I would quite like to spend some time in February or so working on our >>>>>>> tests. >>>>>> >>>>>> Would it make sense to use openQA for this? Qubes OS uses openQA >>>>>> and it works very well. openQA is written in Perl, but it’s the >>>>>> best tool I know of for this. >>>>> >>>>> First blocker there would be packaging openQA in Nixpkgs. I do not >>>>> personally relish the idea of doing that. >>>> >>>> Would it be possible to instead use a Fedora container? openQA is >>>> packaged in Fedora. Qubes OS uses dedicated CI machines for openQA, >>>> so I'm not worried about whether this would be permitted on your dev >>>> box or the binary cache builders. >>>> >>>> I use Fedora for everything that isn't Spectrum-related dev work, >>>> so I know how to maintain a Fedora system. That said, a container >>>> shouldn't need much (if any) ongoing maintenance. >>> >>> I think the hermicity and bisectability of our build and tests are >>> important properties worth preserving. We lose that if we start relying >>> on an opaque container image. If an openQA update breaks something, >>> it's not possible to easily figure out why. >> Fedora container images contain an RPM database that can be used >> to determine which packages changed. There will likely be many >> packages that changed between images, but the same is true of Nixpkgs. >> I totally agree that using a mutable Fedora system that is upgraded >> in-place would be a mistake. > > This is not sufficient for bisectability, because I have no access to > intermediate steps between the two images.
How is Nixpkgs better in this regard? Is it because Nixpkgs only changes one package at a time and has a linear history? Question for the Qubes developers: has Qubes OS ever ran into a regression in openQA itself, and how hard was it to debug? -- Sincerely, Demi Marie Obenour (she/her/hers) -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/qubes-devel/5017a410-0cee-4f15-8eed-dfd836cd42a0%40gmail.com.
OpenPGP_0xB288B55FFF9C22C1.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
