The problem with testing has been argued about. The fact that Nielsen only surveyed his subscribers, most of whom are quite different from Joe User, probably provided different result than if more 'average' testing was done. However, I don't really think that's a bad thing. I think it's actually better that way.
The greatest issue that I have with usability testing and such is that they rely on the flawed assumption that the users know what they want. Don't fool youself. From my experience, if you ask Joe User what an Office program should look like, he'll describe what MS Office looks like, and if you ask what a search engine should look he'll say (in more or less words) "like Google". This happens a *lot* in linux, so much so that usability testing in linux essentially means 'trying to make it look like Windows'. Which is terrible because Linux is not Windows, and to try to make them the same would be a disservice to Linux. The same thing happens on the web (although less so). I read though the report and I saw this in serveral places. Particularly in the search section. Why does everyones search form need to be like Google's? Why do the search results need to be on a seperate page? Having a javascript search function that brough real-time result may lead to an increase in functionality (as long as you have a proper system set up for users without javascript). ****************************************************** The discussion list for http://webstandardsgroup.org/ See http://webstandardsgroup.org/mail/guidelines.cfm for some hints on posting to the list & getting help ******************************************************
