Here are some classic references that discuss usability goals. The earliest examples of usability specifications that I could locate came from Tom Gilb in the late 1970s and early 1980s. Whiteside, Bennett, and Holtblatt's chapter in the Handbook of HCI described usability specifications and highlighted how field work can inform usability goals. Mayhews book describes how goals fit into the usability engineering lifecycle.
Gilb, T. (1988). Principles of software engineering management. Wokingham, England: Addison-Wesley. In his book on software engineering Gilb actually uses "Usability" in some of his examples as a quality attribute of his products and he had principles for developing attribute specifications that include: “measurability” (all attributes should be made measurable) and "result-oriented attributes (the attributes should be specified in terms of the final end-user results demanded). Gilb also gets into principles for choosing solutions to help designers meet those objectives. Mayhew, D. (1999). The usability engineering lifecycle: A practitioner’s handbook for user interface design. San Francisco. CA: Morgan Kaufmann. Mayhew’s book is a detailed blueprint of the usability engineering life cycle with a wealth of practical advice. This book has four sections: Requirements Analysis, Design/Testing/Development, Installation, and Organizational Issues. Each chapter discusses usability engineering tasks, roles, resources, levels of effort, short cuts (quick and dirty techniques to use when a rigorous approach isn’t possible), Web notes, and sample work products and templates. The book is both detailed and readable and worthwhile for both new and experienced usability specialists. Whiteside, J., Bennett, J., & Holtzblatt, K. (1988). Usability engineering: Our experience and evolution. In M. Helander, (Ed.), Handbook of human-computer interaction (pp. 791-817). Amsterdam: North-Holland. This chapter laid out the general guidelines for a usability specification which is the deliverable listing a product’s "usability requirements". A usability specification contains the usability attributes that are critical to the product's quality, the technique for measuring the attributes (which would include the context, constraints, user data requirements, etc), the quantitative metric that represents the usability value (say task completion rate without assistance), and the minimum level of usability for each attribute and the planned level. The Whiteside, et. al. chapter also made a point that usability requirements (and the scenarios for obtaining usability requirements) should be based on field input (through contextual inquiry or other methods) so that the requirements are realistic. Wixon, D. & Wilson, C. E. The Usability Engineering Framework for Product Design and Evaluation. Handbook of Human-Computer Interaction (Second Edition). Elsevier: Amsterdam, The Netherlands, 1998, pp. 653-688. Scott brought up some good issues about the impact of metrics. If you metric is around finding problems and you are not persuasive enough to get them implemented then your impact might be low. Paul Sawyer, Dennis Wixon and Alicia Flanders wrote about a metric they called the impact ratio. Here is the reference and abstract Sawyer, P., Flanders, A., and Wixon, D. 1996. Making a difference—the impact of inspections. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Common Ground (Vancouver, British Columbia, Canada, April 13 - 18, 1996). M. J. Tauber, Ed. CHI '96. ACM, New York, NY, 376-382. DOI= http://doi.acm.org/10.1145/238386.238579 ABSTRACT In this methodology paper we define a metric we call impact ratio. We use this ratio to measure the effectiveness of inspections and other evaluative techniques in getting usability improvements into products. We inspected ten commercial software products and achieved an average impact ratio of 78%. We discuss factors affecting this ratio and its value in helping us to appraise usability engineering's impact on products. So this metric gets at how many are implemented, but there is another step - how much did the changes that were implemented improve the product on whatever usability attributes are most important. What if you implement fixes for 80% of the problems, but the fixes are bad. So, perhaps you can measure how fixes from one version to the next make the product better but does it impact the revenues/profits of the company. It could be that you made your product 20% better and met your goal, but your competitor just came out with a really usable and useful product and was 20% better than your version. So the link between goals and revenues is often really hard to figure. You might want to read some of the chapters in the Bias and Mayhew book Cost-Justifying Usability (Second Edition, 2005). http://www.amazon.com/Cost-Justifying-Usability-Second-Interactive-Technologies/dp/0120958112/ref=sr_1_1?ie=UTF8&qid=1236384677&sr=1-1 Chauncey On Fri, Mar 6, 2009 at 4:24 PM, Alan Cox <[email protected]> wrote: > As it grows, the company I work for is becoming more metric-driven. > Ultimately, I support the idea of having goals and metrics that help > us understand whether we're doing good work, the right work, etc. > > I don't expect goals & metrics to ever tell the whole story; the > world is squishy and numbers are unlikely to paint a completely > honest picture. I do think, however, that they'll help us start > conversations and give us something to shoot towards. > > I'm curious: what type of goals and metrics exist in your company > that are related to good user experience and good design? Do you > have goals & metrics that are company-wide, team-wide and > individual? > > Alan > ________________________________________________________________ > Welcome to the Interaction Design Association (IxDA)! > To post to this list ....... [email protected] > Unsubscribe ................ http://www.ixda.org/unsubscribe > List Guidelines ............ http://www.ixda.org/guidelines > List Help .................. http://www.ixda.org/help > ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [email protected] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
