At 12:42 01/10/02 -0700, you wrote:
Has anyone redesigned the search engine on their institutional website and conducted formal usability studies of the results?
Well, sort of. On theclearances.org site we log the parameters and terms used in the various search tools on the site and analysis of these (though not the full usability gamut) illustrated a lot of weaknesses in the search tools: free text search, even for a narrowly defined (by the site content) domain produced off-the-wall search requests for other time periods and geographical areas; free text search on locations showed lots of spelling variations and predicated either heavy background matching algorithms or less freedom of choice in the seach and so on. What it highlights most is what we know already, search engines, in the conventional Google-like sense, are not necessarily the best, and certainly not the only, interface required to a museum site. Given that most non-curatorial visitors to museums couldn't care less about your collection management system that is hardly surprising. The trick is to provide tools which allow visitors to ask the type of questions they want and to recognise there are many different 'audiences', sometimes an audience of one.
That leads to the question of the value of usability studies. From what people were saying at the MCN conference I got the impression that museums were doing usability studies because they felt they had to measure something and there is plenty of literature out there on usability. It also ties in with Jakob Nielsen as the one-size fits all design guru. Usability studies, and the user-centred design, is all about task analysis, user memory maps, system metaphors for clearly defined groups. Once the group becomes diffuse it becomes harder to generalise about what sort of memory maps users are making (if any). Similarly all Nielsen's Web stuff comes from his 1980s work on hypertext where he talks about mapping working best for small information spaces. A website to sell soap powder is a small information space: an online museum presence should not be.
Of course, that leaves the question of if not usability studies, what. CHIN have been talking about their stickiness quotient (or similar, I forget), time spent by user x number of return visits which has some obvious problems but is a step in the right direction. The measure of success is going to depend on your job spec and your museum's mission statement. It may be the objective of the website is to decrease 'phone calls to the information desk - easily measured; it may be to increase visitors to the physical space - put some discount coupon on the site for people to bring with them - again easily measured.
There may be more subtle social objectives, increasing community awareness for example, which are harder to measure. Just because they are harder doesn't mean they are impossible. When training outcomes of CBT programs were hard to measure we used to ask users how much better they were at their jobs after the training and work from this. I am trying to put a paper together on non-usability evaluation methods for museums (quite far down the 'to do' list) so would be interested in other people's experiences.
Douglas The Highland Clearances http://www.theclearances.org --- You are currently subscribed to mcn_mcn-l as: [email protected] To unsubscribe send a blank email to [email protected]
