Are package uploaders properly testing their own packages? When I wrote a one passphrase/multi volume cryptsetup interface simply to use it myself in systemd and dracut, I had to set up a dummy partition with a keyfile so I could test that option, as otherwise I could not write it into the program and know if it worked or not.
I assume package devs are testing every option they include, or are they writing code they hope will work and packing it up into .debs untested? On 5/21/2014 at 1:07 PM, "Elfy" <[email protected]> wrote: > >Some comments in line ... > >It's likely to get a bit long, sorry about that ;) > >On 19/05/14 10:32, Kaj Ailomaa wrote: >> If anyone is interested in helping out with writing and >performing tests >> during this cycle, please answer this mail (and do read on). >This is the most important bit here to be honest, if there are >only a >/few/ people that would be willing to run package tests then >anything >else is rather, struggling to find a word here that isn't >*pointless* > >When we (and for anyone reading this for the purposes of this mail >- >*we* is Xubuntu QA) started to write our testcases, there wasn't a >huge >crowd of people doing that - it took us a cycle to get the >testcases >written for us. We were then in a position to use those properly >during >the LTS cycle - and it went really well for us. > >Now, our applications are less complicated than many of yours. >Consequently, I'm not going to be able to do much in the way of >helping >to write testcases for you - what I could do - is start setting up >the >barebones of testcases for you, which someone with more experience >of an >application can flesh out. > >They aren't complicated to write - it just gets time consuming and >rather repetitive - certainly not a very glamorous job - but it is >one >that pays dividends in the end. >> ---- >> >> We hardly do any testing at all during our cycle, currently. >This needs >> to be changed. >> >> Naturally, we do required tests for our releases, the Beta >releases and >> the final release, but other than that, there's no structured >testing. >> >> There are two kinds of testing that we would like to do: >> * Quality Assurance Testing - to make sure there are no bugs >for a wide >> range of applications >> * performance testing (which is rather a big topic) >> >> The most urgent type of testing we need to deal with is the >first of >> those. >> >> (So far, what we have in testing documentation can be found here >> https://wiki.ubuntu.com/UbuntuStudio/TestingDocumentation) >> >> # QA testing >> >> I suggest we establish a plan for testing, write test cases, and >such, >> until Debian Import Freeze >(https://wiki.ubuntu.com/DebianImportFreeze), >> which is scheduled to happen Aug 7th this cycle. >> Debian Import Freeze is a great time to do testing on Debian >imported >> packages, since those packages won't be changing before release. >It also >> gives us some time to find bugs, report them and fix them >(Testing can >> of course be done from day one of our development cycle. The >more time >> we have to spot bugs and fix them, the better, but we should >begin no >> later than Debian Import Freeze). >> >> So: >> * Test writing may starts any time >> * Testing of applications should begin no later than at Debian >Import >> Freeze, Aug 7th >I have a suggestion here, why not pick a handful of applications, >get >them landed in the manual testcase branch - then we can set up the >tracker and people can start testing. > >Doing this - people get practice at writing them, people can start >testing as soon as the tracker is up, you start to get results >sooner - >I would think it better to get reported bugs slowly to start with >than >to suddenly have 20 or 30 tests - all being run, all producing >results >at the same time. > >> Elfy has offered to give us a hand on this. If he likes, he >could take >> the role of QA lead for Ubuntu Studio during the next cycle, and >mentor >> us into set up testing. What do you think elfy? >I am more than happy to help you with this goal, there are >probably some >infrastructure issues with the trackers that need to be sorted out >Launchpad wise, if you want me to do that I can talk to Nick >Skaggs >about what needs to be done. > >Let me know if you want me to do that please. > >As I alluded to earlier - 'we' took longer than a cycle - so I'm >happy >to help you all while you need the help, if that's longer than a >cycle - >so be it. > > >> >> The people who write the tests should know the applications they >write >> the tests for. The test should be as simple as possible, but >still >> designed to spot as many typical problems as possible for that >> application. >If anyone wants a look at how testcases are written for the >majority of >cases, then > >bzr branch lp:ubuntu-manual-tests > >and have a look in /testcases/packages/ > >So, those are my thoughts at the moment - feel free to ask me >questions >about how we have worked our system. > >I tend to be about early morning for a while (06:00UTC ish) and >later in >the day 17:00UTC onward for 5 or so hours. > >My IRC nick is elfy - I've also dropped forestpiskie into your - >devel >channel, so if elfy is missing you can ping forestpiskie and I'll >read >it in the morning. > >Obviously I am also on this list and will answer queries etc as >soon as >I can > >Hope that helps you, > >Elfy > >-- >Ubuntu Forum Council Member >Xubuntu QA Lead -- ubuntu-studio-devel mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
