I mentioned in February that I've been working on a new website redesign as a "back-burner" project for a while. I also shared that I was working with undergraduate students in Usability Testing, at the University of Minnesota and Michigan Tech University, to do a usability test of the "new" website.
The students have been working on their usability tests over the semester. This week, the first group (MTU) presented their results. Brief background: The new website design was proposed to us for free by a professional website designer based in Germany. I modified the design slightly (the web designer wanted to change the colors and logo and some other things .. I changed these back) and I used that to create a mostly-working version of the new design at https://test.freedos.org/ The student groups designed, executed, and analyzed a usability test of the test website from late February (or early March?) to mid April. The MTU students presented their results this week. (U of M students will present their results on May 3.) MTU usability test: (my summary) The MTU student group focused their usability test on these scenarios. Each scenario involved one or more tasks: 1. Browse the website to find where to download FreeDOS 2. Find instructions to install FreeDOS 3. Find the FreeDOS wiki (the wiki was not part of the test - that's a whole other cleanup project - but I wanted to know if people could find the wiki) 4. Find how to join the FreeDOS email list 5. Find recent news about FreeDOS MTU used 5 testers (you need a minimum of 5 testers to get feedback that's *good enough* to make design changes .. but we had other groups doing usability tests too) who were between 21-29 years old, and rated themselves as more technical users. Results: (my summary) 1. Browse the website to find where to download FreeDOS (3 tasks) Easy. Testers had little to no problems completing all three of these tasks. 2. Find instructions to install FreeDOS (2 tasks) Hard. Two testers were completely unable to complete either task. The other three testers had trouble finding the install instructions (but found them). 3. Find the FreeDOS wiki (1 task) Very easy. No issues here. 4. Find how to join the FreeDOS email list (1 task) Hard. All testers struggled with this task. 5. Find recent news about FreeDOS (1 task) Easy. Testers had little to no problem completing this task. Additional comments (my summary): Testers thought the website reminded them of other websites about software, so it felt familiar. But testers also commented that the design seemed pretty minimal (this was a mock-up that was 90% complete, so I'm not surprised by that). Testers reported they could usually find what they needed, and the website was fast, and they felt they had an overall "positive" experience on the new website. Recommendations (my summary): * Make installation instructions easier to find: There's a "How to install" link on the front page - this was going to be a button, but it was just a link in the test website. However, testers said they expected the "How to install" link to be on the "Download" page, and they wouldn't have expected it to be on the front page. They also had recommendations to change some other text, like replacing "What you need" with a more direct call to action like "Get started with FreeDOS." * Make the email lists easier to find: Testers said they expected there to be a separate "Email list" link on the website, either at the top of the page or in the footer. They eventually found it in the "Contribute" page. Once they found it there, they said it made sense, but that wasn't where they thought to look for it. * Add a "Home" button in the top navigation bar. Testers said they didn't realize at first that the logo was a link back to the home page. Once they tried clicking on it, they said the "logo as link" made sense, but they would expect to find a link called "Home" somewhere on the page, like at the top of the page. * Use "News" to describe what's new. The test site has a "News" page, so they found it. But I think the recommendation here was that once you clicked on the page, it was called "What's New" and not "News." * Add bullet points in the news page for new software updates. The general comments here are that people don't like reading a lot of text, so they'd prefer just to see text like "IA-16 GCC Toolchain And Libi86 Library, Jan 2022 Version" or "Updated DWED Editor For DOS" as "bullet points" and have some other way to show details. __ My comments: Overall, the results are good. I'll need to update the design again (tweaks, not an overhaul) but I'd planned to do that anyway, so not really surprising. Looks like some of the new design elements work well - but not everything. I won't plan any changes until I hear from the U of M student groups. This feedback from MTU suggests I need to do additional cleanup and reorganization, including "streamlining" or reducing the text, and simplifying navigation and web page content and layout. Again, I was planning to do some of that anyway, so that's not a big surprise. I've asked for the students' permission to share their report on our wiki. If they say Yes, I'll post a text copy there, with charts. If any of them say No, then I'll probably post this summary to the wiki instead. Jim _______________________________________________ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
