Re: [USABILITY] Usability test of Debian Installer beta 3
Hi, step 2-9: (debian-installer) I'm a bit suprised to see the user going through all the help in such detail. This suggests that more online help later on would be a good thing. We're getting that in the new version of partman at least. It also suggests that we should somehow make more clear that you shouldn't need to read all that stuff unless you're curious, or something is going wrong, but I am not sure of how to accomplish that. Yesterday I let a a friend who isn't familar with installations test the new installer. What she found most confusing was, that there was no visible online-help telling her what the key tab does. cursor-keys, esc and enter were intuitive (at least for her), but tab was not. Since the network was broken I then had to fix it and she tried the installer for a second time. This try was much faster and straight-forward, and my test person confirmed that the installer is easy to use if one knows how,. So I got the idea of some initial help screen right after the languagechooser: -- Welcome to the debian-installer! Use the cursor keys, enter, esc and tab to navigate. Select your action: Read more about how to use the installer (recommended for new users) Continue without getting more help. -- Something like this. The second help screen(s) also should tell the user, that almost every question has sane defaults, but the potentially dangerous questions have safe defaults, which you have to change, otherwise no installation will happen. Another thing she found irritating were the information given when installing grub. Since she selected take over the whole disc when partitioning there is no need to worry a user about not to be able to boot other operatings systems and such - simple install the bootloader. I also think it's completly irrelevant for new users to know whether their Powermac-System is a oldworld or new world mac and whether they need yaboot or quik - just make the system boot. But I would like an option when installing the bootloader offering this information - but this should be optional, e.g. install the bootloader and tell me some background infos. Also the ISA network cards were not detected, we tried a smc-ultra and a 3c590 - is this code missing or could it be hardware problem ? Since we unfortunatly used a rather old net-install-cd (2004-03-30) and she will has to repeat the installation onto a bigger harddisc anyway, I won't post any more details now but rather then. But all in all, even though I had to help a bit, the test person had the feeling that she had successfully done (and understood) a debian installation and was happy about that, also acknowleding the fact that d-i is work in progress and has some bugs and todos. regards, Holger step 10: (debian-installer) It should be possible to reword the prompt on the help screens to make clearer that enter boots the system. I'll see what I can do. step 11: (rootskel-floppy) I think you're too charitable; many new users would not like the linux boot text at all. Luckily it's only this in your face on an install from floppies; normally it flashes by and the installer loads in seconds. step 13: (languagechooser) It's a pity that she did not scroll down and find the fi_FI entry. I wonder if adding arrows to the scroll bar, or some other indication that there is more below would have helped her. step 17: (countrychooser) The fact that hitting enter on a continent returns to the menu is something I have always disliked, but I have no particularly better idea. step 18-19: (kbd-chooser) It's probably a bug that the keyboard selector did not default to Finnish here. step 20: (load-floppy) You're right, that needs to stress that now is the time to change the floppy. I'll see about fixing this. step 21: (anna) This mess has been on my list to fix for a while, but it's nontrivial. The retreivers need to be extended to have failure handling capabilities, instead of this generic handler; the failure handler for the floppy should re-prompt for the floppy if none was found. step 22-30: (main-menu) I've seen users get into this kind of confusion when something goes wrong and the priority is lowered. Short of avoiding ever letting things go that badly wrong (which is a noble goal, but perhaps unobtainable), it's hard to fix it. Note that none of this is shown to users unless something goes wrong. On your other comments: - I agree that cdebconf's multiselct box implementation is confusing. Luckily we have no multiselct lists in the standard install path (unless something goes wrong). I would like to see it easier to use, better key assignments or at least a help bar at
Re: [USABILITY] Usability test of Debian Installer beta 3
Frans Pop wrote: I rather like this becase it also deals with the case where downloading installer components fails because of a transient network problem. This will let the user choose to try downloading a component again. It could even let the user choose a new mirror. Ah, that would be very good because I've had installations fail at least once because a local mirror was in the middle of updating and already had the new Packages file, but not all the packages :-( (I still don't understand why they insist on synchronizing during daytime.) Unfortunatly, it won't help with the longer download done by debootstrap. But I'll have it done for anna by tomorrow sometime, just working out the last bugs now. -- see shy jo signature.asc Description: Digital signature
Re: [USABILITY] Usability test of Debian Installer beta 3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2004-04-08 01:01, Joey Hess wrote: step 17: (countrychooser) The fact that hitting enter on a continent returns to the menu is something I have always disliked, but I have no particularly better idea. how about: North America (cursor)- enter - North America (cursor) Canada Central America ... Belize United States ... Central America Panama Belize South America Argentina Panama ... South America Chile Argentina ... Chile i.e having enter on a continent expand/collapse the list of countries for that continent? Does the debconf interface allow this? - -- Cheers, cobaco (aka Bart Cornelis) 1. Encrypted mail preferred (GPG KeyID: 0x86624ABB) 2. Plain-text mail recommended since I move html and double format mails to a low priority folder (they're mainly spam) -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAdRyO5ihPJ4ZiSrsRAr7iAJ9rwGmDLWItKcT06Od7xGiGMrsNJACffzty tAdn1hk2Jq9lOtiGRYIFtvM= =jaQe -END PGP SIGNATURE-
[USABILITY] Usability test of Debian Installer beta 3
Hi, Inspired by the simulated usability review at http://www.thiesen.org/d-i/index.html by [EMAIL PROTECTED], I thought I'd take the usability testing of d-i one step further by arranging a real usability test. Yes, that's with a live crash test dummy. :) I hoped to see a result similar to Marcus' simulation, but unfortunately I didn't. While I've only tested with one person so far, I now believe there is a rather big amount of usability flaws in d-i. Further testing is certainly warranted, but I couldn't set aside the results I've got so far. I'm concentrating on users who have never installed an operating system before. While this may at first seem like the wrong thing to do, I actually think it's a very sensible approach. If a user who has never installed any operating system can use d-i to successfully install Debian, then anyone can. Adding more control and features for experienced users should be easy, but if the installer is that simple then my guess is that even the gurus will appreciate getting the installation over with so they can concentrate on more important things such as installing and configuring software packages. The complete results of the usability test is at http://people.paniq.net/~fabbe/tmp/d-i-test/ (yes, tmp means temporary -- it'll be up for many weeks, even months, though). Feedback is welcome, and I'm subscribed to debian-boot so please discuss here. I'm trying to find time to arrange a few more tests within the next month or so, so that I can see what other flaws emerge. Cheers, -- Fabian Fagerholm [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [USABILITY] Usability test of Debian Installer beta 3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wednesday 07 April 2004 20:06, Fabian Fagerholm wrote: I'm concentrating on users who have never installed an operating system before. If you're going to use complete newbies for testing, I think you should be fair and provide them with relevant information that is freely available. At least provide them with a version of the manual which _does_ explain about scrolling and the sequence of the installation, common problems and how to solve them. Note that the manual is work in progress (just like the installer itself) and will change considerably before the final release. A relatively experienced user will probably try to install Debian without a manual (maybe only referring to it if he/she runs into trouble), but I think a real newbie would make use of a manual if one is available. Currently the manual is only publicly available online in HTML, but I can provide you with a pdf if you mail me privately. Also I think you should use the netinst CD installation method and not the floppy based one for inexperienced users. They will probably buy a CD-set for installation as it is by far the easiest method to get an installation set. You could even help improve the manual by also providing a usability report on that ;-) Cheers, FJP -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFAdE8qgm/Kwh6ICoQRAnEGAKCnHRXLwNF9x9RUV2qsGMGlbm9ixwCfcozf F21eroj9rckPRVkQ2mahivo= =E1Ek -END PGP SIGNATURE-
Re: [USABILITY] Usability test of Debian Installer beta 3
On Wed, Apr 07, 2004 at 08:57:46PM +0200, Frans Pop wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wednesday 07 April 2004 20:06, Fabian Fagerholm wrote: I'm concentrating on users who have never installed an operating system before. If you're going to use complete newbies for testing, I think you should be fair and provide them with relevant information that is freely available. At least provide them with a version of the manual which _does_ explain about scrolling and the sequence of the installation, common problems and how to solve them. [...] No, help screens have to be improved, but unfortunately this will be done very late. Current d-i developers are working on other tasks, so anyone interested in providing a better installer can help, the only prerequisite is to have tested the debian-installer. For instance a help line can be added at the bottom to explain available keystrokes, and I will be glad to incorporate it if there are suggestions. Another useful help would be to test current daily images and see if criticisms expressed in http://www.thiesen.org/d-i/ have been dealt with, and file individual bugs if this has not already been done. Denis -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [USABILITY] Usability test of Debian Installer beta 3
Hello Frans, Thanks for your feedback! On Wed, 2004-04-07 at 21:57, Frans Pop wrote: If you're going to use complete newbies for testing, I think you should be fair and provide them with relevant information that is freely available. At least provide them with a version of the manual which _does_ explain about scrolling and the sequence of the installation, common problems and how to solve them. The manual is not a usability solution. Neither is a troubleshooting section that solves problems caused by bad interaction design. I eliminated the need for the manual and the troubleshooting section by ensuring that the hardware was suitable for d-i beta 3 (that there were no funny drivers required and that the hardware discovery worked). I was testing d-i, not a manual. Had the interface been good enough, the user could have installed the system without problems. Currently the manual is only publicly available online in HTML, but I can provide you with a pdf if you mail me privately. Does the PDF contain the same data as in the SVN repository? If so, I already built the PDF. Also I think you should use the netinst CD installation method and not the floppy based one for inexperienced users. They will probably buy a CD-set for installation as it is by far the easiest method to get an installation set. My plan is to test the netinst CD installation later. I hope I can find enough test users and *time* :) You could even help improve the manual by also providing a usability report on that ;-) I'll see if I can fit in a few test cases where I give the user a printed installation manual in addition to the installation media and see how that affects the results. Cheers, -- Fabian Fagerholm [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [USABILITY] Usability test of Debian Installer beta 3
Fabian Fagerholm wrote: Feedback is welcome, and I'm subscribed to debian-boot so please discuss here. I'm trying to find time to arrange a few more tests within the next month or so, so that I can see what other flaws emerge. Thanks for doing the testng (and please send our thanks to your tester). I really encourage you to do further testing using a CD to install; installing by booting a CDROM will work on the vast majority of hardware, including any hardware a true new user would be likely to have, and it is much less tortuous and error prone. I expect that your tester would have gotten to at least the base-config process if using a CD. By the way, if you do test with CDs, please use full-size CDs, and not the netinst CDs. I'd think it best to break out the individual points in your summary into bug reports on the right installer components, and here's some commentary to that end: step 2-9: (debian-installer) I'm a bit suprised to see the user going through all the help in such detail. This suggests that more online help later on would be a good thing. We're getting that in the new version of partman at least. It also suggests that we should somehow make more clear that you shouldn't need to read all that stuff unless you're curious, or something is going wrong, but I am not sure of how to accomplish that. step 10: (debian-installer) It should be possible to reword the prompt on the help screens to make clearer that enter boots the system. I'll see what I can do. step 11: (rootskel-floppy) I think you're too charitable; many new users would not like the linux boot text at all. Luckily it's only this in your face on an install from floppies; normally it flashes by and the installer loads in seconds. step 13: (languagechooser) It's a pity that she did not scroll down and find the fi_FI entry. I wonder if adding arrows to the scroll bar, or some other indication that there is more below would have helped her. step 17: (countrychooser) The fact that hitting enter on a continent returns to the menu is something I have always disliked, but I have no particularly better idea. step 18-19: (kbd-chooser) It's probably a bug that the keyboard selector did not default to Finnish here. step 20: (load-floppy) You're right, that needs to stress that now is the time to change the floppy. I'll see about fixing this. step 21: (anna) This mess has been on my list to fix for a while, but it's nontrivial. The retreivers need to be extended to have failure handling capabilities, instead of this generic handler; the failure handler for the floppy should re-prompt for the floppy if none was found. step 22-30: (main-menu) I've seen users get into this kind of confusion when something goes wrong and the priority is lowered. Short of avoiding ever letting things go that badly wrong (which is a noble goal, but perhaps unobtainable), it's hard to fix it. Note that none of this is shown to users unless something goes wrong. On your other comments: - I agree that cdebconf's multiselct box implementation is confusing. Luckily we have no multiselct lists in the standard install path (unless something goes wrong). I would like to see it easier to use, better key assignments or at least a help bar at the bottom. (cdebconf) - Some of your UI ideas are not particularly doable with the current simple queston and answer, cdebconf-based interface. Some of them can be approached in spirit, if not in actual UI. (cdebconf) - Your idea for presenting a list of information the installer needs (language, country, keyboard) is a good one, and it's similar to an earlier proposal I made, which would also include some other information prompted for later. Unfortunatly, it will be a lot of work to implement this, and I've put off working on my idea until after the first release. There is still time to do it, I think, if someone is interested. Plan for a good week's work. (main-menu) - Indeed we don't display the kernel modules screen on normal installs. We do display a progress bar that breifly mentions the modules that are being loaded; while this will contain terms that users are not familiar with, I think the overall thing is clear, they do not have to interact with it, and it's essential for debugging when a module freezes the machine. (hw-detect) As to your conclusions, I think that you're jumping to conclusions from one test with floppies. Out of our 300+ installation reports (and the many more users who have installed without reporting), I think there are a few that are from users nearly as novice as your tester, and succeeded. However, they all used CDROMs. I look
Re: [USABILITY] Usability test of Debian Installer beta 3
Joey Hess wrote: step 10: (debian-installer) It should be possible to reword the prompt on the help screens to make clearer that enter boots the system. I'll see what I can do. Done. step 21: (anna) This mess has been on my list to fix for a while, but it's nontrivial. The retreivers need to be extended to have failure handling capabilities, instead of this generic handler; the failure handler for the floppy should re-prompt for the floppy if none was found. I took another look at it, and it's not that hard. My current implementaton adds a new command to the retriever spec: retriever error failing_command This command can be called if one of the other commands seems to have failed, perhaps by returning bad data. The failing_command field tells the retriever command that previously failed. For example, it might be packages is package retrieval failed, or the Packages file cannot be parsed. The retriever can take any necessary actions to recover (or ask the user if they want to retry, if the medium is unreliable or absent). If the recovery succeeds, it should exit 0. If recovery failed, it should display an informative error message if necessary, and exit 2. Retievers should implement this command, but temporarily for backwards compatability, if a retriever exits 1, it is assumed that it did not support this command, and anna will display its own (ugly) error message. I have an implementation for the floppy retreiver and the particular error that's generated when there is not a driver floppy in the drive. If this looks ok, we will need to add the error command to all the other retrievers eventually, and add support for the other possible errors in anna. I rather like this becase it also deals with the case where downloading installer components fails because of a transient network problem. This will let the user choose to try downloading a component again. It could even let the user choose a new mirror. -- see shy jo signature.asc Description: Digital signature
Re: [USABILITY] Usability test of Debian Installer beta 3
I rather like this becase it also deals with the case where downloading installer components fails because of a transient network problem. This will let the user choose to try downloading a component again. It could even let the user choose a new mirror. Ah, that would be very good because I've had installations fail at least once because a local mirror was in the middle of updating and already had the new Packages file, but not all the packages :-( (I still don't understand why they insist on synchronizing during daytime.) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
