A week ago Friday we held a test session in our OSAF offices in San
Francisco. We had ~10 volunteers, drawn from the other small
organizations that share office space with us. (We call this group Kapor
Klein Inter Entites, or KKIE).
We discussed the planning of the session on the general list, but I'm
moving the summary over to the design list, as this discussion is
primarily a product discussion.
http://lists.osafoundation.org/pipermail/general/2007-August/000953.html
We were primarily interested in testing the documentation
(chandlerproject.org, download pages, Get Started Guide, Screencasts,
etc.) and how it helped (or didn't help) with a first time user's
initial experience. We asked people to download the desktop, create an
account on the server, and try sharing a collection. We didn't get
around to the screencasts.
Info given to the testers:
http://chandlerproject.org/Projects/KKIETestSession
The test session was perhaps flawed as a "usability" test, as we didn't
get a chance to coach OSAF folks who helped run the test session. OSAF
helpers did a lot of explaining, less observing and asking questions
about what people were thinking. That said, we got a lot of interesting
and useful feedback.
Some testers sent notes to chandler-users@, others sent notes to Aparna
and me directly. All of the notes I have are here:
http://chandlerproject.org/Projects/KKIETestSessionNotes20070817
The notes contain many detailed suggestions that I thought were best
tracked as bugs, which I went ahead and logged (these are indicated in
the notes). One of the benefits of getting new eyes on the applications
is that they find these kinds of bugs and problems.
Some observations from the session (some captured in the notes and some
from looking over people's shoulders):
+ People found the download/install experience on the mac more confusing
than I expected.
+ People published calendars without much grief, but the "invite" task
was a little more tricky. Partly this could have been a consequence of
the way we gave instructions, but I think this is an area we should
focus on after Preview. Wording changes in dialogs and menus could help
somewhat, as could breaking the documentation down into bite size
pieces. A specific change to the docs was to mark the "Invite" section
more clearly (I think Pieter already did this). We could also look at
more expensive changes to the workflow.
+ No one wanted to sit down and read all of the documentation in one go.
Pieter and I discussed taking the "Get Started Guide" back to the
initial idea -- focused instructions for specific tasks. People should
be able to make sense of each section without reading the whole guide.
+ Several people wanted to see a preferences panel in the desktop.
+ Both desktop and web ui felt stable, testers were successful at
sharing collections (with a little help). People enjoyed the test session.
Cheers,
Katie
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Design" mailing list
http://lists.osafoundation.org/mailman/listinfo/design
