Hi Folks, Based on some comments and a short exchange on the dev list, I've developed a rough draft for a UX survey. The goal is to solicit user feedback about how usable the AAO site is. To help frame the results and grasp who the users are, several qualitative and demographic questions are included.
I'm brand new to the AOO community, so I apologize if I've overlooked something obvious. Below is the proposed survey. I have brief explanatory comments between each small set of questions. I do hope this is useful or provides a viable foundation for a future UX review. Best, gz *********************** *********************** Proposed Core SUS Usability Questions to Evaluate User Experience with the Apache Open Office Website [Draft] The purpose of this text is to: propose a body of questions for use in a usability test of the AOO site; provide support and explanation for these questions; identify important issues to help us frame the data. To measure user experience and user response to the AOO website, it is important to use reliable tools to gather data. Fortunately, a variety of such tools exist. One optimal tool is the System Usability Scale which John Brooke released in 1986. The SUS is a 10 item questionnaire with 5 response options. SUS Questions I think that I would like to use this system frequently. I found the system unnecessarily complex. I thought the system was easy to use. I think that I would need the support of a technical person to be able to use this system. I found the various functions in this system were well integrated. I thought there was too much inconsistency in this system. I would imagine that most people would learn to use this system very quickly. I found the system very cumbersome to use. I felt very confident using the system. I needed to learn a lot of things before I could get going with this system. Response options: 1 Strongly Disagree to 5 Strongly Agree source: http://www.measuringusability.com/sus.php [A number of other sources can be references, and SUS is an industry standard.] SUS Strengths: The system is tested and is reliable. The data produced is pretty clean, and there are numerous parties able to analyze and evaluate the data. The brevity is useful for users because it does not take too much of their time. It takes far less time to identify, code, and gather Likert-type response than open-ended questions. SUS Weaknesses: Results produced only enable system-wide results, i.e. it does not help identify specific or local problems. Results are strictly numerical. Users are not able to give voice to their own opinions or share accurate or useful insights--user can only click a box. Proposed Addition to SUS: In addition to the SUS scale, I propose having four or five additional qualitative questions which allow users to give more rich, personalized, specific, or custom feedback. This can be useful in terms of identifying specific problems, demonstrating interest in users’ opinions, and identifying gaps that those in the AOO culture are too close to to identify. These kinds of questions, however, can generate huge volumes of responses. And often they are different. Evaluating and coding this information can drive you nuts. One way to address this potential problem is to frame the open-ended questions and then provide tick boxes for the three or five most common responses. Additional Questions: What do you like most about the Apache Open Office website? <options> The forums The online manuals ...add a couple more x user fills in their own response What frustrates you most about the Apache Open Office website? <options> Navigation The help manuals ...add a couple more x user fills in their own response If there was one thing you could add to the Apache Open Office website, what would it be? <options> ...add a couple more x user fills in their own response Potential transitional text for users: Thank you for providing your responses and your feedback. Your responses will help us make AOO a better and stronger community. If you are willing, we would appreciate a bit more information about you, your computing preferences, and how you use AOO. Additional Demographic Questions We probably want to keep these limited, otherwise they get tedious and drive users away. Also, we should include a “prefer not to answer” as an option for each of the questions. Gender: F M prefer to not answer Age range: 18-22 23-30 31-35 36-40 41-45 46-50 51-55 56-60 61-70 70+ prefer to not answer Operating system: Windows Mac Linux Other prefer to not answer Browser: Firefox Chrome Safari IE Other prefer to not answer Version of Open Office: prefer to not answer Industry in which you work: Government Education Industry Entrepreneur Creative prefer to not answer Where you are located (continent? country?) dropdown menu of options? prefer to not answer Language in which you use AOO? dropdown menu of options? prefer to not answer More Transitional text for users: Again, thank you for sharing your time, thoughts and responses. As you know, AOO is as committed to open source as we are to privacy. We are also interested in being as transparent with our growth, processes, and data as possible. With your permission, we would like to publish and share your responses with our community. We won’t share it without your permission. To remain anonymous and not share your data, you don’t need to do anything else. But if you are willing to share your responses, please tick the box below. Sharing Data Checkbox: I agree to share my data/responses. [And whatever legalese is needed.] -- --- Gregory B. Zobel, Ph.D. Assistant Professor of Educational Technology MSEd Program Coordinator Western Oregon University 345 N. Monmouth Ave Monmouth, OR 97361
