Craig Errey wrote: > Certified Usable is backed by our Professional Indemnity insurance and as > such is a specifically name service on our policy. For an industry that is > generally risk averse, they have audited what and how we're doing it and are > comfortable to include it, with no effect on our premium.
So is it a way for the corporate bean counters to "feel secure" with the product, in that light ? > Regarding the statement that essentially goes: '90% success, within x minutes > +/- 10%', we must use ranges like this because we cannot guarantee 100% > usability. Because we use strong statistical methods in the process, we must > use confidence intervals and make our statements framed with such statistical > rigour and various caveats. This is what gives our insurer confidence that > we are not doing things that cannot be done, and that would expose them to > risk. > The process is being run by Psychologists (Registered in NSW) who have an > extensive understanding of testing design, rigour and statistical analysis to > ensure the process is run correctly and defensibly. In fact, the ethics that > psychologists must abide by regarding testing procedures means that we are > fully accountable for what we do by the NSW State Government (specifically > Department of Health). There is no pseudo-science here, and no illusion of > competency. These staff have at least a Masters degree in Psychology and 5 - > 10 years experience in usability. Usability ? As in human cognitive development or HCI or both ? Which ? > Regarding what we're defining as usable, it is entirely task driven. It is > strictly not designed for simple sites or marketing sites, but can be used > for them. Rather it is geared at complex transaction sites and rich > applications like internet banking, online travel booking, ERP and CRM > systems. In these sites, success rates and time taken (the two primary > measures) are easily measured. So if the task is quick and achieves a "success", even if the work flow doesn't make intuitive sense, it's "Certified Usable" ? (This is related to the perils of some CMS solutions I've had the pleasure of working with. What may be usable to a software engineer may not be usable elsewhere. Go read Jeffrey Veen at length on this..) > Regarding setting the standard and testing for it, because the process is > task driven, and relies on user testing, the evidence for task success or > failure, or time taken is observable and can be independently verified. It's > a bit like Standards Australia providing consulting on ISO 9000 and then > auditing you for compliance. Although they did not necessarily set the > original standard, there is independent proof of whether a company complies > or not and it is not their opinion, even though they may have set up the ISO > process in the organisation. Pass or fail is independently verifiable and > another testing group would come to the same conclusion. Task driven solutions sometimes do not make intuitive sense, no matter how fast they are. Which gets into the whole field of "self-fulfilling prophecy" in that the "product" works efficiently "because it does". > In the case of Certified Usable, we set a tough benchmark, usually at a > minimum 90% success. We do not use an easy target, such as 50% success rate. > There would be no point as real world performance would not match the claim > of Certified Usable. Yet from what it looks like at first glance, CU is simply a way for the corporate bean counters to "feel secure" about a product so that they/you have a potential inside run against the multitude of CMS solutions out there. > There is no conflict of interest because the measurement of achieving the set > standard is transparent. It is not our opinion because it is based on > observable behaviour - someone's success is usually binary, and the time take > is finite. As psychologists, we are good at observing and documenting > behaviour and doing it in a way that is accurate and repeatable. So where's the associated peer reviewed psychological studies of such things ? Transparency should mean that others who likewise have studied human cognitive behaviour within the workplace can test whether your tools actually do the job. (And yes, I'd be one of them : that is my only caveat within this discussion, acknowledging the potential for conflict of interest) What may be usable to a deaf user may not be usable to a blind user or vice versa, to paraphrase of the team leaders I've been working with in terms of accessible CMS solutions in the last few years. lawrence -- Lawrence Meckan Absalom Media Mob: (04) 1047 9633 ABN: 49 286 495 792 http://www.absalom.biz ****************************************************** The discussion list for http://webstandardsgroup.org/ See http://webstandardsgroup.org/mail/guidelines.cfm for some hints on posting to the list & getting help ******************************************************
