Re: Default Desktop Experience for 11.04 - User testing results

2011-04-20 Thread Jan Claeys 
Jeremy Bicha schreef op di 19-04-2011 om 15:37 [-0400]:
> On 19 April 2011 15:13, Jan Claeys  wrote:
> > Matthew Paul Thomas schreef op di 19-04-2011 om 15:55 [+0100]:
> >> *   The object of the study was, obviously, not to measure crashes.
> >> Crashes are usually quick to find and fix, so any user test of those
> >> would be weeks out of date when published. I mentioned them only as
> >> a reminder that to users, bugs are indistinguishable from design
> >> flaws, and vice versa. (For example, one test participant pressed
> >> Ctrl Alt F1 apparently by accident, and ended up at a console. This
> >> wasn't a crash, but it had exactly the same effect as one.)
> >
> > Maybe we need to add a line of text above the login prompt somehow, that
> > tells the user what key to press to get back to their GUI?  (This might
> > be difficult to do correctly with multiple logins etc. though?)
> 
> We could do like Fedora and have X on virtual terminal 1. Why do we
> need 6 virtual terminals anyway? 

I'm sure that's a possibility, but you still need more than 1 VT anyway,
e.g. when more than one user is logged in.  And developers might need
the virtual text consoles, while maybe their machine is also used by
inexperienced people at other times.

Maybe it would be better to just disable all the Alt+Ctrl+Fx keys by
default, with an option to enable them in the keyboard configuration
(similar to what was done with Ctrl+Alt+Backspace)?


-- 
Jan Claeys


-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-19 Thread IKT 
because screen is hard mode.

On 20 April 2011 05:07, Jeremy Bicha  wrote:

> On 19 April 2011 15:13, Jan Claeys  wrote:
> > Matthew Paul Thomas schreef op di 19-04-2011 om 15:55 [+0100]:
> >> *   The object of the study was, obviously, not to measure crashes.
> >> Crashes are usually quick to find and fix, so any user test of those
> >> would be weeks out of date when published. I mentioned them only as
> >> a reminder that to users, bugs are indistinguishable from design
> >> flaws, and vice versa. (For example, one test participant pressed
> >> Ctrl Alt F1 apparently by accident, and ended up at a console. This
> >> wasn't a crash, but it had exactly the same effect as one.)
> >
> > Maybe we need to add a line of text above the login prompt somehow, that
> > tells the user what key to press to get back to their GUI?  (This might
> > be difficult to do correctly with multiple logins etc. though?)
>
> We could do like Fedora and have X on virtual terminal 1. Why do we
> need 6 virtual terminals anyway?
>
> Jeremy
>
> --
> ubuntu-desktop mailing list
> ubuntu-desktop@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop
>
-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-19 Thread Jeremy Bicha 
On 19 April 2011 15:13, Jan Claeys  wrote:
> Matthew Paul Thomas schreef op di 19-04-2011 om 15:55 [+0100]:
>> *   The object of the study was, obviously, not to measure crashes.
>>     Crashes are usually quick to find and fix, so any user test of those
>>     would be weeks out of date when published. I mentioned them only as
>>     a reminder that to users, bugs are indistinguishable from design
>>     flaws, and vice versa. (For example, one test participant pressed
>>     Ctrl Alt F1 apparently by accident, and ended up at a console. This
>>     wasn't a crash, but it had exactly the same effect as one.)
>
> Maybe we need to add a line of text above the login prompt somehow, that
> tells the user what key to press to get back to their GUI?  (This might
> be difficult to do correctly with multiple logins etc. though?)

We could do like Fedora and have X on virtual terminal 1. Why do we
need 6 virtual terminals anyway?

Jeremy

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-19 Thread Jan Claeys 
Matthew Paul Thomas schreef op di 19-04-2011 om 15:55 [+0100]:
> *   The object of the study was, obviously, not to measure crashes.
> Crashes are usually quick to find and fix, so any user test of those
> would be weeks out of date when published. I mentioned them only as
> a reminder that to users, bugs are indistinguishable from design
> flaws, and vice versa. (For example, one test participant pressed
> Ctrl Alt F1 apparently by accident, and ended up at a console. This
> wasn't a crash, but it had exactly the same effect as one.) 

Maybe we need to add a line of text above the login prompt somehow, that
tells the user what key to press to get back to their GUI?  (This might
be difficult to do correctly with multiple logins etc. though?)


-- 
Jan Claeys


-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-19 Thread Matthew Paul Thomas 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jono Bacon wrote on 16/04/11 20:05:
>
> On Fri, 2011-04-15 at 03:00 +0100, Matthew Paul Thomas wrote:
>...
>> Last week, Charline Poirier ran a user test of Unity, with 11
>> individual participants. This week, I have helped Charline analyze the
>> results.
>...
> Wonderful work, and now very visible work:
> http://news.slashdot.org/story/11/04/16/0239213/5-Out-of-11-Crashed-Unity-In-Canonicals-Study
>  ;-)

Up to Slashdot's usual standards, I see.

To correct the obvious from that post:

*   No, "the results" of the study have not been published. Charline
will do that soon.

*   No, my name is not Rick Spencer. (rickspencer4?)

*   The object of the study was, obviously, not to measure crashes.
Crashes are usually quick to find and fix, so any user test of those
would be weeks out of date when published. I mentioned them only as
a reminder that to users, bugs are indistinguishable from design
flaws, and vice versa. (For example, one test participant pressed
Ctrl Alt F1 apparently by accident, and ended up at a console. This
wasn't a crash, but it had exactly the same effect as one.)

> I think this feedback points to a series of design and engineering bugs
> that we need to resolve in 11.10. Have the design bugs been filed in
> Launchpad?

Charline has been working with John Lea on that today.

> I think it could be worthwhile to rate the prioritization of the design
> bugs based upon the level of success in your study. As an example, if
> 1/12 achieved a task, it would be a high priority bug, as opposed to if
> 10/12 achieved the task it would be a low priority bug.
>...

I'll pass that on to John.

Cheers
- -- 
mpt
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2tokcACgkQ6PUxNfU6ecr1TgCePApLBm/ySLUAasdjINBrWQHs
LKYAoNLXk9mfWvVzz9fF7h4rl3eBq9u3
=j4k8
-END PGP SIGNATURE-

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Missing panel items (was Re: Default Desktop Experience for 11.04 - User testing results)

2011-04-18 Thread Alin-Andrei 
On Mon, Apr 18, 2011 at 17:03, Barry Warsaw  wrote:

>>>* The weather notifier is missing.  I really like this little notifier so
I
>>>know when to throw open the windows and get some fresh air!
>>
>>Install the "indicator-weather" package - it's in the official Natty repos
>>(universe).
>
>Thanks, done.  Now what?  I don't know how to add indicators to the Unity
top
>panel.  System settings has nothing appropriate, and right-clicking on the
>panel does nothing.

> Please note, I'm deliberately trying to wear a newbie hat.


Just search for "weather" in Dash... it should show up and click in will
automatically add Weather Indicator to the panel. I guess that for a newbie
it's easier then before as you launch it from the menu and you don't have to
right click the panel to add it. But not for one who has used an older
Ubuntu version of course... for such a newbie I can see how that's
confusing.


>>>* System monitor.  I like this little applet because it gives me a quick
>>>way
>>>to take the pulse of my system.   How hard it's running, is there a lot
of
>>>network activity going on, etc.
>>>
>>>For the CPU and memory usage you can use System Monitor Indicator [1]

>>>* Force quit.  Some times you just need it and a kill -9 just isn't easy
to
>>>get to.
>>
>>There's no appindicator for this, but you can create a .desktop file (just
>>copy any from /usr/share/applications/ on your desktop and edit it as a
>>template), and under "Exec" enter "xkill", then drag and drop this
.desktop
>>file to the Unity launcher. Then to force an application to close, click
>>this icon, then click the window you want to close.
>>
>>I hope this helps :)

>It does, and thanks.  But do understand that from a non-technical user's
>perspective, these are regressions from the classic desktop, both in
missing
>features and the ability to add them to the top panel.

I'm just a simple user and I though I'd help so you don't need to convince
me but the developers :)



-Andrew
http://www.webupd8.org
-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Missing panel items (was Re: Default Desktop Experience for 11.04 - User testing results)

2011-04-18 Thread Alin-Andrei 
Hello,

>* The weather notifier is missing.  I really like this little notifier so I
>know when to throw open the windows and get some fresh air!

Install the "indicator-weather" package - it's in the official Natty repos
(universe).

>* System monitor.  I like this little applet because it gives me a quick
way
>to take the pulse of my system.   How hard it's running, is there a lot of
>network activity going on, etc.

For the CPU and memory usage you can use System Monitor Indicator [1]

>* Force quit.  Some times you just need it and a kill -9 just isn't easy to
>get to.

There's no appindicator for this, but you can create a .desktop file (just
copy any from /usr/share/applications/ on your desktop and edit it as a
template), and under "Exec" enter "xkill", then drag and drop this .desktop
file to the Unity launcher. Then to force an application to close, click
this icon, then click the window you want to close.


I hope this helps :)



[1]
http://www.webupd8.org/2011/03/system-monitor-indicator-puts-cpu-and.html


-Andrew


On Mon, Apr 18, 2011 at 16:02, Barry Warsaw  wrote:

> Now that I'm using Unity a lot, I find that I'm missing a few things from
> the
> classic desktop's upper panel.  I don't know how to get these back, if it's
> even possible.
>
> * The weather notifier is missing.  I really like this little notifier so I
>  know when to throw open the windows and get some fresh air!
>
> * System monitor.  I like this little applet because it gives me a quick
> way
>  to take the pulse of my system.   How hard it's running, is there a lot of
>  network activity going on, etc.
>
> * Force quit.  Some times you just need it and a kill -9 just isn't easy to
>  get to.
>
> Cheers,
> -Barry
>
> --
> ubuntu-devel mailing list
> ubuntu-de...@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>
>
-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-17 Thread Rick Spencer 
On Sun, 2011-04-17 at 09:17 +0100, Paul Sladen wrote:
> On Fri, 15 Apr 2011, Rick Spencer wrote:
> > > > For brand new users? Some of the tasks aren't relevant.
> > > Which ones?
> > Changing the background image and setting in general.
> 
> On the contrary.  Changing the wallpaper and screensaver are comforting
> personalisation steps that new users seem to undertake on their own.
That was my point. You don't personalize something until you *own* it.
Until you've used it enough that you know how you want to own it. It's
not something you do while you are carrying out other tasks.

In any case, it's a moot point, since changing the desktop background
was something that the study found users could do easily, they were
blocked by the usability engineer telling them not to do it in the
obvious manner.

Cheers, Rick




-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-16 Thread Jono Bacon 
On Fri, 2011-04-15 at 03:00 +0100, Matthew Paul Thomas wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Rick Spencer wrote on 08/04/11 02:38:
> >...
> > Back at UDS for 11.04 in Orlando, Mark set the goal of using Unity by
> > default on the Ubutu desktop. Given the current course of development,
> > it appears that we are going to achieve this goal, and Unity will stay
> > the default for 11.04.
> > 
> > I'm following up on this list at the suggestion of the Tech Board to
> > give folks a chance to respond or escelate any concerns.
> >...
> 
> Last week, Charline Poirier ran a user test of Unity, with 11 individual
> participants. This week, I have helped Charline analyze the results.



Wonderful work, and now very visible work:
http://news.slashdot.org/story/11/04/16/0239213/5-Out-of-11-Crashed-Unity-In-Canonicals-Study
 ;-)

I think this feedback points to a series of design and engineering bugs
that we need to resolve in 11.10. Have the design bugs been filed in
Launchpad?

I think it could be worthwhile to rate the prioritization of the design
bugs based upon the level of success in your study. As an example, if
1/12 achieved a task, it would be a high priority bug, as opposed to if
10/12 achieved the task it would be a low priority bug.

I just want to ensure that we build this feedback into the design and
engineering iteration process.

Jono


-- 
Jono Bacon
Ubuntu Community Manager
jono(at)ubuntu(dot)com
www.ubuntu.com : www.jonobacon.org
www.twitter.com/jonobacon : www.identi.ca/jonobacon


-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Rick Spencer 
On Fri, 2011-04-15 at 23:29 +0100, Matthew Paul Thomas wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Rick Spencer wrote on 15/04/11 15:08:
> >
> > First, thanks Usability Team! I know how much work goes into planning
> > and running a study like this, and how much agony is involved in
> > interpreting and writing up the results. It's clear that there are some
> > areas for improvement in 11.10, and these results will be instrumental
> > in helping to guide those investments.
> 
> Charline did all the planning and test moderation. I was just the
> stenographer afterwards.
> 
> Later on, Charline will publish a full report on the test. I just wanted
> to post a quick summary in time to be helpful for the default experience
> discussion.
> 
> > On Thu, 2011-04-14 at 22:48 -0700, Bryce Harrington wrote:
> >>
> >> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> >>>
> >>> *   8/10 people could find a window's menus, but 7/8 of them learned to
> >>> *   Only 4/11 worked out how to change the background picture.
> >>> *   6/10 could easily find and launch a game that wasn't in the
> >>> *   Only 1/9 (P4) easily added that game to the launcher.
> >>> *   9/11 people could easily close a window.
> >>> *   8/9 easily copied text from one document into another.
> >>> *   Only 5/10 could easily delete a document
> >>
> >> These seven items in particular seem like really basic tasks that
> >> ought to be testing at >90%, so these stats seem a lot lower than I'd
> >> expect.
> 
> > Well, think back to the last time you got a new device. For example, if
> > you have an Android phone. You are probably pretty facile with the
> > interface now, but if someone handed it to you and said "do this task
> > with it" you may have struggled to some basic things, like launching
> > apps. A lot of the fun for users in getting a new devices is learning
> > how to use it.
> 
> I didn't have anything close to that kind of trouble when trying out an
> Android phone. (Though like anyone on a developer mailing list, I'm not
> a representative sample.)
Oh? I recall learning many things on my phone. How to launch apps. How
to use the task manager to switch between running apps. How to enter
text in general and swype in particular. How to pull down the
notification area and how to push it up again, etc... If you were
watching over my shoulder you may have said "Fail Fail Fail" many times.
Perhaps you would have been correct, the usabliity could be better.
However, it's easy to forget the first few times you satisficed through
something compared to the hundreds of times you've done something after
you mastered it.

> 
> >...
> > For brand new users? Some of the tasks aren't relevant.
> 
> Which ones?
I believe the following are tasks indicative of repeated and experienced
usage, not first time usage:
 * Changing the background image and setting in general.
 * Adding a game to the launcher.
 * Rearranging items in the launcher.

And these were some that users had the most trouble with.

Cheers, Rick


-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Matthew Paul Thomas 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rick Spencer wrote on 15/04/11 15:08:
>
> First, thanks Usability Team! I know how much work goes into planning
> and running a study like this, and how much agony is involved in
> interpreting and writing up the results. It's clear that there are some
> areas for improvement in 11.10, and these results will be instrumental
> in helping to guide those investments.

Charline did all the planning and test moderation. I was just the
stenographer afterwards.

Later on, Charline will publish a full report on the test. I just wanted
to post a quick summary in time to be helpful for the default experience
discussion.

> On Thu, 2011-04-14 at 22:48 -0700, Bryce Harrington wrote:
>>
>> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
>>>
>>> *   8/10 people could find a window's menus, but 7/8 of them learned to
>>> *   Only 4/11 worked out how to change the background picture.
>>> *   6/10 could easily find and launch a game that wasn't in the
>>> *   Only 1/9 (P4) easily added that game to the launcher.
>>> *   9/11 people could easily close a window.
>>> *   8/9 easily copied text from one document into another.
>>> *   Only 5/10 could easily delete a document
>>
>> These seven items in particular seem like really basic tasks that
>> ought to be testing at >90%, so these stats seem a lot lower than I'd
>> expect.

> Well, think back to the last time you got a new device. For example, if
> you have an Android phone. You are probably pretty facile with the
> interface now, but if someone handed it to you and said "do this task
> with it" you may have struggled to some basic things, like launching
> apps. A lot of the fun for users in getting a new devices is learning
> how to use it.

I didn't have anything close to that kind of trouble when trying out an
Android phone. (Though like anyone on a developer mailing list, I'm not
a representative sample.)

>...
> For brand new users? Some of the tasks aren't relevant.

Which ones?

>...
>> Also, these tests measure usability, but not their overall impression.
>> Did they like it?  Find it frustrating/confusing?
>
> This is actually a very important question. For me, when I go back to
> Classic, it feels very old and mundane. Many theories hold that the
> aesthetics trump usability, or at least strongly influence the
> perception of the usability of a system. In other words, given 2
> identically design systems that only differ in terms of theming, for
> example, users will rate the system with the more pleasing design to be
> more "usable" and are more likely to start using it. 
>...

This is the aesthetic usability effect.



- -- 
mpt
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2oxq0ACgkQ6PUxNfU6ecoIcACcCcdaRX0LStM999wHOWz+q/GU
4YEAn05ZbWUmv1dPOkBJzHvpYb8eSaXg
=I/jZ
-END PGP SIGNATURE-

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Bryce Harrington 
On Fri, Apr 15, 2011 at 01:33:27PM +0100, Matthew Paul Thomas wrote:
> Bryce Harrington wrote on 15/04/11 06:48:
> > On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> > Also, these tests measure usability, but not their overall impression.
> > Did they like it?  Find it frustrating/confusing?
> >...
> 
> In my limited experience, user test participants can often be highly
> complimentary about an interface that completely failed them (or
> critical of an interface where they had no trouble). They can also start
> guessing about what other people what other people might think, or
> guessing about how easily they might learn something later.

Good point.  Even so I find the comments on their impressions adds a lot
of context to the numbers, so thanks muchly for forwarding them.

AIUI, ensuring a good initial impression was one of the major goals for
the new design work, and judging from the comments this was definitely
achieved.  Most everyone had positive comments as to the aesthetics.
My favorite is: "So I want to play with it now."

Of the negative comments, the theme seems to be around learnability,
which may also be a quirk of the testing process (people may have felt
pressure to do the given tasks quickly so they can complete the test).
But this issue has also gotten a fair bit of airing on blogs and mailing
lists.  I've heard several sources also make P4's suggestion about
making demo videos easily available.

Jorge put out a demo video on how to multitask in Unity, which I found
very engaging and enlightening.  Ideally stuff should be discoverable,
but until it is maybe an effective workaround would be to just ensure
demo videos are easily at hand?  Like a hyperlink to an online
collection of videos that's provided in the global help menu, on the
desktop, or on the welcome page of the webbrowser or something.

Bryce

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Bryce Harrington 
On Fri, Apr 15, 2011 at 07:08:02AM -0700, Rick Spencer wrote:
> On Thu, 2011-04-14 at 22:48 -0700, Bryce Harrington wrote:
> > On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> > > *   8/10 people could find a window's menus, but 7/8 of them learned to
> > > *   Only 4/11 worked out how to change the background picture.
> > > *   6/10 could easily find and launch a game that wasn't in the
> > > *   Only 1/9 (P4) easily added that game to the launcher.
> > > *   9/11 people could easily close a window.
> > > *   8/9 easily copied text from one document into another.
> > > *   Only 5/10 could easily delete a document
> > 
> > These seven items in particular seem like really basic tasks that ought
> > to be testing at >90%, so these stats seem a lot lower than I'd expect.
> Well, think back to the last time you got a new device.

No, I definitely agree there, but this is why I suggested a control
group - to help understand if it's just ordinary newbie level learning
curve stuff.  Like, maybe with Classic Desktop only 2/10 new users could
find a new game (esp. if they're equating USC with the recycling bin...)
so 6/10 with unity would be a huge step forward.  On the other hand if
with Classic it's at 9/10 then I think you'd be right that it's an area
needing further attention.

In any case, I'm loving the numbers, you know how much I geek out on
quantification of issues.  :-)

I would love to see this same test repeated again every so often, so we
can chart the results and use them to demonstrate that we're getting
better.

mpt's additional info adds a lot more context to the numbers and makes
them sound a lot less troublesome, but I'll comment against that one
separately.

Bryce

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Toby Smithe 
2011/4/15 Matthew Paul Thomas :
> *   8/10 people could find a window's menus, but 7/8 of them learned to
>    access them by hovering over maximized close/minimize/unmaximize
>    buttons then moving horizontally -- which was extremely slow, and
>    failed whenever the window wasn't maximized.

Why not swap the default state of the panel from one where the
application name is shown in full and the menus hidden, to the
alternative where the menus are shown, and long application names are
partially obscured? Then, allow that if someone mouse-overs the
application name, it reveals itself fully; in this case, an arrow
symbol (for example) could be included to make this action
discoverable.

I believe this swap wouldn't cause any usability problems (unless
knowing the long name of a window is critically important), but
instead would make the location of window menus obvious.

Of course, if it is intentional that menus are hard to discover (to
push application developers towards Chrome-like streamlining), or if
there would be problems due to claims of Mac-mimicry, then it clearly
makes sense to keep the current behaviour.

-- 
Toby Smithe
http://launchpad.net/~tsmithe

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Rick Spencer 
First, thanks Usability Team! I know how much work goes into planning
and running a study like this, and how much agony is involved in
interpreting and writing up the results. It's clear that there are some
areas for improvement in 11.10, and these results will be instrumental
in helping to guide those investments.

On Thu, 2011-04-14 at 22:48 -0700, Bryce Harrington wrote:
> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> > *   8/10 people could find a window's menus, but 7/8 of them learned to
> > *   Only 4/11 worked out how to change the background picture.
> > *   6/10 could easily find and launch a game that wasn't in the
> > *   Only 1/9 (P4) easily added that game to the launcher.
> > *   9/11 people could easily close a window.
> > *   8/9 easily copied text from one document into another.
> > *   Only 5/10 could easily delete a document
> 
> These seven items in particular seem like really basic tasks that ought
> to be testing at >90%, so these stats seem a lot lower than I'd expect.
Well, think back to the last time you got a new device. For example, if
you have an Android phone. You are probably pretty facile with the
interface now, but if someone handed it to you and said "do this task
with it" you may have struggled to some basic things, like launching
apps. A lot of the fun for users in getting a new devices is learning
how to use it.

Given my experience, these numbers look they could be improved, but I
don't find them particularly concerning. For example, you can easily use
Unity quite productively before you learn that you can add items to the
launcher, or change the background picture, though I suspect many
Windows users would right-click and the desktop to set the background
picture and would do fine.

The only area here that is at all concerning to me regards launching
applications. I'd like to see some focus on the application lens in
11.10 (see what I did I did there? :) )

> 
> I'm curious whether these stats are higher/lower/same-as with Classic
> Desktop.  IOW this needs a control group so we can tell if the new
> design brings improvement or regression.
For brand new users? Some of the tasks aren't relevant. For others, like
finding installed applications, I presume this was dead easy in Classic
GNOME, but it's hard to say for a certain.

> 
> Also, these tests measure usability, but not their overall impression.
> Did they like it?  Find it frustrating/confusing?
This is actually a very important question. For me, when I go back to
Classic, it feels very old and mundane. Many theories hold that the
aesthetics trump usability, or at least strongly influence the
perception of the usability of a system. In other words, given 2
identically design systems that only differ in terms of theming, for
example, users will rate the system with the more pleasing design to be
more "usable" and are more likely to start using it. 

Cheers, Rick


-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Matthew Paul Thomas 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bryce Harrington wrote on 15/04/11 06:48:
>
> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
>>
>> *   8/10 people could find a window's menus, but 7/8 of them learned to
>> *   Only 4/11 worked out how to change the background picture.
>> *   6/10 could easily find and launch a game that wasn't in the
>> *   Only 1/9 (P4) easily added that game to the launcher.
>> *   9/11 people could easily close a window.
>> *   8/9 easily copied text from one document into another.
>> *   Only 5/10 could easily delete a document
> 
> These seven items in particular seem like really basic tasks that ought
> to be testing at >90%, so these stats seem a lot lower than I'd expect.
> 
> I'm curious whether these stats are higher/lower/same-as with Classic
> Desktop.  IOW this needs a control group so we can tell if the new
> design brings improvement or regression.

Sure, that's the main problem with using this data. The test was
intended to find fixable problems in Unity, not to compare Unity with
Classic. And the tests Canonical has done on Ubuntu previously have
mostly been on other tasks, such as importing photos and music and
burning CDs.

> Also, these tests measure usability, but not their overall impression.
> Did they like it?  Find it frustrating/confusing?
>...

In my limited experience, user test participants can often be highly
complimentary about an interface that completely failed them (or
critical of an interface where they had no trouble). They can also start
guessing about what other people what other people might think, or
guessing about how easily they might learn something later.

That said, here's a representative sample.

P1: "I've got a Mac, and I think it's quite similar to a Mac, the
layout, which made it easier to use. I think people going from Windows
to this would need a little help, because it's a very different layout."
And: "It's a nice casual-ish font that you've used here."

P2: "I'm guessing the way it's set at the moment, with the wallpaper,
the background, I'd personally want everything to be crisper and clearer
and more obvious ... I find it all a bit blending into each other, with
those settings. But I do like the overall design of it, with the nice
curves and the nice icons at the side, and the font used, and the design
of it's really nice."

P3: "It's prettier than a lot of current operating systems." And: "I
don't think it's a complicated operating system ... it would just take
time."

P4: "Some of the features, like when you minimize something you don't
find the menu bar, so it's quite hard to find." And: "The simple is
quite simple, not too hard to follow or too hard to understand. If
possible, if you could give them a demo. This is where you go to get a
file, this is where you get all your icons, for the first time user,
obviously."

P5: "I really quite like it. I think it's quite intuitive, with the
exception of the favorites, making an application a favorite, which
obviously I wasn't able to achieve. I wouldn't be baffled about how to
use this operating system for the first time, if I didn't have a manual
to read ... It's quite clean-looking, it's quite modern-looking. It
seems to me to be closer to a Mac-style operating system than to a
Microsoft-style operating system."

P7: "No. I don't know. So at this point, I would have to say my initial
impressions are -- I wouldn't write it off, because I've heard too many
good things about it, but my initial impressions are 'Damn, I'm going to
need a manual' ... I don't really want to do that, which is a bit lazy
of me, I know." At the end: "It's not really working for me. I'm finding
it time-consuming and slightly confusing at times ... That's my kind of
bottom-line impression, really, that the differential [between Windows
and Ubuntu], the gap is really very big."

P8: "Maybe the settings should have been more prominent, but I found
them eventually. Fairly standard. If it can run anything I wanted, I
would consider it."

P9: "It's okay ... It's not as confusing as the Mac."

P10: "So generally I think it's pretty good. I think it needs some work
on it. I think making it a bit more intuitive."

P11: "Oh yikes, is it like Apple? ... I never liked the [Mac] interface.
I didn't know where I stored things, where I put things. I like these
pictures. It's aesthetically pleasing. It also looks like it's
child-friendly, because it's pictures. And it looks more ordered than
Apple. So I want to play with it now." At the end: "I think it's pretty.
It's aesthetically pleasing."

P12: "And I really like the cleanness of it all. It's tidy." At the end:
"It's quick and responsive. I'm still frustrated that I'm not able to
find anything like My Computer. And hardware as well, I want to see what
the hardware is, the versions of the graphics and drivers."

- -- 
mpt
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mo

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-15 Thread Celeste Lyn Paul 
On Fri, Apr 15, 2011 at 1:48 AM, Bryce Harrington  wrote:
> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
>> *   8/10 people could find a window's menus, but 7/8 of them learned to
>> *   Only 4/11 worked out how to change the background picture.
>> *   6/10 could easily find and launch a game that wasn't in the
>> *   Only 1/9 (P4) easily added that game to the launcher.
>> *   9/11 people could easily close a window.
>> *   8/9 easily copied text from one document into another.
>> *   Only 5/10 could easily delete a document
>
> These seven items in particular seem like really basic tasks that ought
> to be testing at >90%, so these stats seem a lot lower than I'd expect.

What would be telling is if the people who didn't figure out the task
the first time remembered how to complete it the next time. That shows
that the task is learnable, which is acceptable if the curve matches
the difficulty of the functionality. Unfortunately very rarely do you
use the same users for multiple tests (in fact it is usually
discouraged unless you are doing longitudinal studies) except in
longitudinal studies.

>
> I'm curious whether these stats are higher/lower/same-as with Classic
> Desktop.  IOW this needs a control group so we can tell if the new
> design brings improvement or regression.
>
> Also, these tests measure usability, but not their overall impression.
> Did they like it?  Find it frustrating/confusing?

Do people in the UK use the System Usability Scale, NASA TLX, or
Modified-Cooper Harper? Theyre assessment surveys that measure
satisfaction and cognitive load, but I've only seen human factors
engineers use them.

>
> Bryce
>
> --
> ubuntu-desktop mailing list
> ubuntu-desktop@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop
>



-- 
Celeste Lyn Paul
KDE Usability Project
KDE e.V. Board of Directors
www.kde.org

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-14 Thread Bryce Harrington 
On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> *   8/10 people could find a window's menus, but 7/8 of them learned to
> *   Only 4/11 worked out how to change the background picture.
> *   6/10 could easily find and launch a game that wasn't in the
> *   Only 1/9 (P4) easily added that game to the launcher.
> *   9/11 people could easily close a window.
> *   8/9 easily copied text from one document into another.
> *   Only 5/10 could easily delete a document

These seven items in particular seem like really basic tasks that ought
to be testing at >90%, so these stats seem a lot lower than I'd expect.

I'm curious whether these stats are higher/lower/same-as with Classic
Desktop.  IOW this needs a control group so we can tell if the new
design brings improvement or regression.

Also, these tests measure usability, but not their overall impression.
Did they like it?  Find it frustrating/confusing?

Bryce

-- 
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Re: Default Desktop Experience for 11.04 - User testing results

2011-04-14 Thread Matthew Paul Thomas 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rick Spencer wrote on 08/04/11 02:38:
>...
> Back at UDS for 11.04 in Orlando, Mark set the goal of using Unity by
> default on the Ubutu desktop. Given the current course of development,
> it appears that we are going to achieve this goal, and Unity will stay
> the default for 11.04.
> 
> I'm following up on this list at the suggestion of the Tech Board to
> give folks a chance to respond or escelate any concerns.
>...

Last week, Charline Poirier ran a user test of Unity, with 11 individual
participants. This week, I have helped Charline analyze the results.

In this summary, I have numbered the participants:
- - P1, 19, a student and Mac user
- - P2, 33, an administrator and Mac user
- - P3, 25, a student and Windows user
- - P4, 32, a teacher and Windows user
- - P5, 27, a compliance officer who uses both Windows and Mac
- - P7, 44, a life coach who uses both Windows and Ubuntu
- - P8, 30, an IT network manager and Windows user
- - P9, 22, a student and Windows user
- - P10, 21, a student and Windows user
- - P11, 47, a teacher and Windows user
- - P12, 34, an operations manager and Windows user.

The test machine was a Lenovo ThinkPad T410i running Ubuntu Natty with
unity 3.8.2-0ubuntu1 and compiz 1:0.9.4git20110322-0ubuntu5.

Charline asked each participant to try several tasks. Not every
participant had time to try every task.

*   Every participant who was asked understood most of the launcher
items. P7 and P11 thought that "LibreOffice Calc" was a calculator,
and P7 and P9 thought Ubuntu Software Center was the Recycle Bin.
Nobody understood Ubuntu One. (The Classic session has much smaller
icons for everything, but has a visible-by-default label plus an
extra tooltip for each app.)

*   Almost everyone understood most of the indicators, but 4/11 people
(P7, P9, P11, P12) thought the Me menu icon might be a close button.

*   11/11 people easily launched Firefox to check Web mail.

*   8/10 people could find a window's menus, but 7/8 of them learned to
access them by hovering over maximized close/minimize/unmaximize
buttons then moving horizontally -- which was extremely slow, and
failed whenever the window wasn't maximized.

*   10/11 worked out how to open a new Firefox window, though 5/10 first
tried clicking Firefox in the launcher again, which didn't work.

*   Only 4/11 worked out how to change the background picture. This is
not as bad as it looks: for some of the others, Charline had asked
them *not* to right-click on the desktop, because she was testing
access to settings in general. Nevertheless, no-one found System
Settings, in the session menu or anywhere else.

*   Only 5/11 could easily rearrange items in the launcher. For the
other six, the main problems were that the launcher scrolled when
they were trying to drag an item, or that it didn't accept a drop.
(P10 was particularly unlucky in doing a Dash search for "menu" and
finding the Main Menu editor, which is useless in Unity but still
present by default.)

*   6/10 could easily find and launch a game that wasn't in the
launcher. (P2 deserves special mention for finding and launching the
game's .desktop file amongst piles of detritus in Nautilus's "File
System" search results.)

*   Only 1/9 (P4) easily added that game to the launcher. For the other
eight, the main problems were that the launcher disappeared when a
window was maximized or at the left of the screen, that Dash items
didn't have context menus, and that the launcher often didn't accept
a drop.

*   2/2 successfully removed an item from the launcher.

*   Only 2/6 noticed an XChat Gnome notification, despite (1) a
notification bubble appearing, (2) the Ubuntu button going blue,
(3) the messaging menu envelope going blue, and (4) an emblem
appearing on XChat Gnome's launcher.

*   9/9 easily launched LibreOffice Writer to write a letter.

*   5/5 easily found today's date.

*   9/9 easily saved their LibreOffice Writer document. (P1 recovered
amazingly well after trying to save "Letter to Mr Smith 08/04/11",
and getting the vile response "Error stating file '/home/ubuntu
/Documents/Letter to Mr Smith 08/04': No such file or directory").

*   9/11 people could easily close a window. The other two (P2, P7)
were not the only ones to be attacked by a bug that hid the title
bar for a window underneath the menu bar; they were just the only
participants for whom that bug really cramped their style.

*   9/9 easily found and opened an existing document.

*   8/9 easily copied text from one document into another. The other one
(P2) managed it eventually, but the missing title bar for one of the
document windows was again a major stumbling block.

*   Only 1/7 (P9, a Windows 7 user) easily arranged windows side by side
by discovering the window snap feature. (That's probably not reall