Some comments in line ...
It's likely to get a bit long, sorry about that ;)
On 19/05/14 10:32, Kaj Ailomaa wrote:
If anyone is interested in helping out with writing and performing tests
during this cycle, please answer this mail (and do read on).
This is the most important bit here to be honest, if there are only a
/few/ people that would be willing to run package tests then anything
else is rather, struggling to find a word here that isn't *pointless*
When we (and for anyone reading this for the purposes of this mail -
*we* is Xubuntu QA) started to write our testcases, there wasn't a huge
crowd of people doing that - it took us a cycle to get the testcases
written for us. We were then in a position to use those properly during
the LTS cycle - and it went really well for us.
Now, our applications are less complicated than many of yours.
Consequently, I'm not going to be able to do much in the way of helping
to write testcases for you - what I could do - is start setting up the
barebones of testcases for you, which someone with more experience of an
application can flesh out.
They aren't complicated to write - it just gets time consuming and
rather repetitive - certainly not a very glamorous job - but it is one
that pays dividends in the end.
----
We hardly do any testing at all during our cycle, currently. This needs
to be changed.
Naturally, we do required tests for our releases, the Beta releases and
the final release, but other than that, there's no structured testing.
There are two kinds of testing that we would like to do:
* Quality Assurance Testing - to make sure there are no bugs for a wide
range of applications
* performance testing (which is rather a big topic)
The most urgent type of testing we need to deal with is the first of
those.
(So far, what we have in testing documentation can be found here
https://wiki.ubuntu.com/UbuntuStudio/TestingDocumentation)
# QA testing
I suggest we establish a plan for testing, write test cases, and such,
until Debian Import Freeze (https://wiki.ubuntu.com/DebianImportFreeze),
which is scheduled to happen Aug 7th this cycle.
Debian Import Freeze is a great time to do testing on Debian imported
packages, since those packages won't be changing before release. It also
gives us some time to find bugs, report them and fix them (Testing can
of course be done from day one of our development cycle. The more time
we have to spot bugs and fix them, the better, but we should begin no
later than Debian Import Freeze).
So:
* Test writing may starts any time
* Testing of applications should begin no later than at Debian Import
Freeze, Aug 7th
I have a suggestion here, why not pick a handful of applications, get
them landed in the manual testcase branch - then we can set up the
tracker and people can start testing.
Doing this - people get practice at writing them, people can start
testing as soon as the tracker is up, you start to get results sooner -
I would think it better to get reported bugs slowly to start with than
to suddenly have 20 or 30 tests - all being run, all producing results
at the same time.
Elfy has offered to give us a hand on this. If he likes, he could take
the role of QA lead for Ubuntu Studio during the next cycle, and mentor
us into set up testing. What do you think elfy?
I am more than happy to help you with this goal, there are probably some
infrastructure issues with the trackers that need to be sorted out
Launchpad wise, if you want me to do that I can talk to Nick Skaggs
about what needs to be done.
Let me know if you want me to do that please.
As I alluded to earlier - 'we' took longer than a cycle - so I'm happy
to help you all while you need the help, if that's longer than a cycle -
so be it.
The people who write the tests should know the applications they write
the tests for. The test should be as simple as possible, but still
designed to spot as many typical problems as possible for that
application.
If anyone wants a look at how testcases are written for the majority of
cases, then
bzr branch lp:ubuntu-manual-tests
and have a look in /testcases/packages/
So, those are my thoughts at the moment - feel free to ask me questions
about how we have worked our system.
I tend to be about early morning for a while (06:00UTC ish) and later in
the day 17:00UTC onward for 5 or so hours.
My IRC nick is elfy - I've also dropped forestpiskie into your -devel
channel, so if elfy is missing you can ping forestpiskie and I'll read
it in the morning.
Obviously I am also on this list and will answer queries etc as soon as
I can
Hope that helps you,
Elfy
--
Ubuntu Forum Council Member
Xubuntu QA Lead
--
ubuntu-studio-devel mailing list
[email protected]
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
