Hi Aparna,
Some thoughts in line...
On Feb 23, 2007, at 12:18 AM, Aparna Kadakia wrote:
Preview release, as we all know, is a big milestone release for
OSAF. Obviously, our quality standards are going to be very high
for this release. As a 4 member QA team, divided between Cosmo and
Chandler, we are grossly understaffed for doing detailed,
exhaustive testing of all the new features. Without going on a mass
hiring spree, we would like to brainstorm on ideas about how we
could collectively as a team commit ourselves to raising the
quality standards for this release.
Some of the things we think would help are:
1. More automation
Add more automated tests to the test suite. This would especially
help in regression testing of existing features by reducing manual
cycles. This also includes adding unit tests for all the new
features being checked in. So when we hit Feature Complete
deadline, it would mean all the unit tests for the specific feature
are also done. These would be run continously on tinderboxes and
thereby help catch any regressions.
I agree that more automation will help in some areas. I would like
to get to a point where Cosmo developers could run windmill tests
against their changes before those changes get checked in. When I
worked on Chandler, I used to run the functional tests before I
checked in a unit of work, and it saved me from embarrassment many a
time.
One area where automation will not help is in interop testing between
Chandler and Cosmo. This is an are where we could do with some more
testing by humans.
2. More pariticipation in the IRC QA sessions
As you all know we now have a dedicated IRC office hour for QA
sessions on wednesdays at 11:00 AM PST. Traditionally only the QA
team, PPD team and the dev managers have been participating in
these sessions. What we would like is to see is more participation
from other team members, especially in cross-product and cross-
feature testing. Hopefully this is not asking for too much. Just
like we make time for staff meetings, design sessions etc, we
should make time for this in our schedule and commit ourselves to
exercising the app in different ways and testing new features to
help uncover bugs.
At an hour a week, this is not a big of a time commitment.
3. More internal dogfooding
If we could just get people internally to download Chandler
periodically and use it for maintaining their personal calendars as
well as updating the office calendar that would be a big help.
I think that we should now be in a situation where everyone can
either use Chandler or Cosmo to update the office calendar. I am
maintaining my work calendar using Chandler, and I will be posting a
Cosmo bookmarkable URL so that people can view/edit calendars. I am
going to stop replying to e-mail requests for my availability and
start replying with that bookmarkable URL.
The other suggestions we have heard are:
1. Maybe increasing the frequency of the IRC QA sessions from once
a week to twice a week with partipation from all prouct teams.
2. All the teams dedicate 1 day a week to testing after feature freeze
3. Hiring a contractor till preview ( I am not sure how much value
this would add)
We would like to hear more suggestions if you have any that have
worked in your previous companies.
A few things to remember that the bugs in the software will turn up
eventually and we will need to fix all of them before the release.
So it would be most efficient on everyone's time in finding them
earlier than later. Also when you cross test features, you bring
fresh perspectives on usability of the feature, uncovering any
design bugs earlier on, if any. This would be valuable user
feedback for the PPD team as well.
At the end of the day we need to release quality software and if
that means we need to get more creative and figure out ways in
which we can leverage the strength of the team for getting there I
sure hope we can do that.
In many open source projects, the developers also spend time testing
the software. I don't see why we can't do the same.
Ted
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
