Re: Hardware compatibility test: draft proposal
On Fri, 2008-08-22 at 11:05 +0100, Chris Lamb wrote: Wouter Verhelst wrote: As I mentioned in my blog[1], I kindof like the suggestion that Bdale came up with during Debconf that we write a hardware compatibility test of sorts that hardware vendors could run on their own hardware I think this is a great idea, and I generally agree with your decisions and assumptions. - Scripts should not just test for availability of hardware. Instead, they should test the actual functionality; e.g., tests for a network interface should be done by trying to DHCP off that interface, X.org drivers should try to start X and ask for input using a graphical dialog, and tests for a hard disk should be done by trying to read some data from the disk. I think this the most important paragraph of all. What I think is missing is some really concrete info on just how various hardware items would be tested. For example, in the case of ethernet adaptors, I feel that simply successfully DHCP-ing on an interface is really not an acceptable test. snip At a bare minimum the HCT should try to run the driver's offline self-test (ethtool -t $dev offline). For a more comprehensive (and generic) test we would really want to use two machines, though an external loopback cable may suffice. My employer outsources much of the generic network test development to Oktet Labs. Their network test environment http://www.oktetlabs.ru/test_env.rhtml is distributed under GPL though it is not freely downloadable. It might be worth asking them to provide it to Debian. Ben. -- Ben Hutchings It is easier to write an incorrect program than to understand a correct one. signature.asc Description: This is a digitally signed message part
Re: Hardware compatibility test: draft proposal
Giacomo A. Catenazzi wrote: Wouter Verhelst wrote: On Fri, Aug 22, 2008 at 12:55:34PM +0200, Giacomo Catenazzi wrote: Wouter Verhelst wrote: I'm not so sure that the test should be packages. The system is too different on early boot from a Debian system. In what way? Note that a Debian Live system *is* a normal Debian system; you build a Debian Live image by giving the debian-live script a bunch of packages; it then installs them into a chroot, changes a few things so that the system will actually boot from CD-ROM, and then create an ISO image from it. How is that too different from a Debian system? Or are you talking about something else? ok. I was thinking about doing hardware test a lot earlier (i.e. in initramfs). But probably it is not really needed, and we probably could assume that system is enough reasonable. Anyway, we can try with debian-live, and ev. move things earlier. The modular thing should be put mainly on the testing development side. Yeah, I'm not suggesting we provide vendors with a bunch of packages; the packages thing is a way for us to make it easier to actually develop this without having one maintainer who has to write/accept/maintain *all* the tests in the system. But hey, this is debian-devel, not debian-announce ;-) - Vendors who pass this test on a particular bit of hardware are allowed to advertise that fact; it might be nice to have provide them with a logo that they may use for this purpose. This should be discussed after we have comprehensive tests, Sure. and probably to LI. I don't want to see packages with logo of 10 distributions. To me, it's not about seeing a logo on a package. It's about seeing another vendor besides HP make a commitment that Yes, Debian will be supported. We won't run away scared to the hills if you say that you're running Debian. Because that's what most vendors will do currently. I think you'll also find aving a generic Linux hardware compatibility test isn't very useful; if we provide hardware vendors with such a thing, they'll go I ran that test, and it said 'pass'; but my shiny-new-network-card-chipset-that's-supported-in-the-latest-development-kernel-only isn't supported in Debian? That's probably a bug in Debian, then. And it doesn't even have to be too new stuff; it may also be that some particular bit of hardware is supported by some free out-of-tree driver which is supported by some distributions, but not necessarily by all of them. Would you test for that in a generic Linux compatibility test? If you don't, you'll fail on an amount of hardware that works perfectly well in most distributions; and if you do, you'll succeed on hardware that won't be supported by a number of distributions, possibly the one the hardware vendor cares about most (because, say, they use it themselves for their webservers). No, I think we *do* need a Debian-specific test. The LinuxFirmwareKit has few kernels. IIRC redhat, suse and latest vanilla, so vendors could check the support of various distributions. I think we could be done in the same way: we allow installing/selecting other kernels. The tests are the same, only the kernel could change, and obviously, the default kernel is the Debian stable kernel. In this manner, vendor could test new-hardware with requires new-kernel but on other side, vendors have information about Debian kernel, so they could ev. try to backport drivers for us. I think doing it Debian specific is more difficult, because it requires a lot more work. we should share some burden to upstreams (I'm thinking mainly about xorg: test about 2D, 3D accelerations, etc.; i2c, etc.). ciao cate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Hardware compatibility test: draft proposal
Joe Smith wrote: Giacomo Catenazzi [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Wouter Verhelst wrote: Hi, As I mentioned in my blog[1], I kindof like the suggestion that Bdale Yes, I find the talk very interesting. So, after more than twelve hours of boredom on an airplane and half a night of not-being-able-to-sleep-due-to-jetlag, which is certainly enough to think about this problem, I came up with the following things such a system could need: /me too - It should support a notion of what I'll call 'profiles'. A 'server' profile should check for different things than a 'desktop' or 'laptop' profile; e.g., it's usually okay if a server doesn't have graphical support or wireless drivers, while the same isn't true for a laptop. No. I don't think we should provide profiles. The check should be: complete: test all hardware that it is installed on system. If the system doesn't have the video-card or the wifi just it doesn't test it, but it write a notice. Listening the available hardware is pretty trivial (see i.e. my AutoKernConf). Ok. What does the test do if it notices that a pecie of hardware exists, but cannot identify it? Should it warn about that? I think that 100% (but maybe very few exceptions) of the new hardware can be detected and identify the class of hardware. Possible problem could arise with some USB gadget, but class list is frequently updated, and in this case we must warn the user about unsupported hardware. (the same with firewire, PCI, and some other buses). Perhaps a few of those could be merged into other tests, like a single optical drive test, but in some of those cases it would then be important that a list of hardware that was found is printed. That way the manufacturer can be sure that both optical drives (in a two optical drive PC) were found. I agree, but probably we should ask Bdale about how the manufacturers will do the test and what we can expect about it. If there is any hardware that cannot be tested with only a single test script due to major implementation differences between all brands of hardware (and no current unified software interface), we would want to avoid having a notice printed for each possible manufacture of the device. Hmm. If hardware cannot be fully detected (e.g. in this case: if we cannot detect what kind of the protocol is used), it is an error, which should be notified, so to correct the test, the driver or the hardware. [i.e. the burden of hardware identification should not be left to our users]. The handle of such cases IMHO should not have a high priority. Probably not always it is possible to do such identification (IIRC recently on kernel and Linus had such problem with Intel e1000 variant), but I think it is rare. ciao cate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Hardware compatibility test: draft proposal
Wouter Verhelst wrote: On Fri, Aug 22, 2008 at 12:55:34PM +0200, Giacomo Catenazzi wrote: Wouter Verhelst wrote: Hi, As I mentioned in my blog[1], I kindof like the suggestion that Bdale Yes, I find the talk very interesting. So, after more than twelve hours of boredom on an airplane and half a night of not-being-able-to-sleep-due-to-jetlag, which is certainly enough to think about this problem, I came up with the following things such a system could need: /me too - It should support a notion of what I'll call 'profiles'. A 'server' profile should check for different things than a 'desktop' or 'laptop' profile; e.g., it's usually okay if a server doesn't have graphical support or wireless drivers, while the same isn't true for a laptop. No. I don't think we should provide profiles. The check should be: complete: test all hardware that it is installed on system. If the system doesn't have the video-card or the wifi just it doesn't test it, but it write a notice. That will create ambiguity, which I think we should avoid at all cost. If you tell a user of this test that it seems to work, but you should check this and this and this output to make sure we didn't miss anything, then such a test is worthless; hardware vendors *will* miss this kind of thing. I really think there are no such cases. Eventually we found unknown hardware, but I really think that on modern hardware we will not be undetected hardware (but on very new bus, in which case the hardware vendor care much more). Listening the available hardware is pretty trivial (see i.e. my AutoKernConf). I'm afraid I'm not familiar with this piece of software; and as I'm offline right now, I can't look it up, either. However, I don't think checking by listing the hardware is the best way to go, in any regard. A script that checks for all hardware available in a system can't know about each and every piece of hardware out there; especially not if new interfaces are made that the script doesn't yet know about. Giving a piece of hardware a perfect score because our script failed to notice there's some unsupported wireless interface connected is not the way to go. I meant the kdetect script, which list the hardware attached to various buses. You are right, the other program try to give a name and a kernel driver to hardware, which is impossible to do completely. But you can list the hardware of pretty all buses, which usually some information about the type. My script collect work of kernel and other programs. Additionally dmidecode collects a lot more information about memory, cache, motherboard, etc, and there is already some program to find sensors (my script find only the sensor found by the kernel, the other program is kernel-indipendent). Now there is also a fan-control script that check the fan (but maybe using dmidecode or i2c) I really think we can detect nearly all hardware. The hardware that could be missed, it is the hardware that probably the vendor will forget (sensor, fan and other small and probably fast changing piece of hardware). Additionally a component test should be furnished, for component hardware vendor. (e.g. a network card vendor would test only the network card, not the whole system). I'm not sure that's very useful. A network card vendor has two options: either they design a card based on an existing chipset that has drivers already in the kernel (in which case it will probably just work, unless they use previously-unknown PCI IDs, in which case it won't), or they will create a totally new chipset for their network card, in which case it won't work. A vendor is usually aware of this fact, and giving them a test for something they already know about isn't very useful. If you disagree, of course don't let me stand in your way. Just don't expect me to work on it ;-) Maybe we should discuss with Bdale. I'm not sure about your reasoning, for two reasons: - hardware vendor will do a new version, but it still use old driver (USB and PCI have an additional version value, along the ID) - a lot of drivers support multiple hardware: if you read the kernel, you will see driver with a long list of ID. Some drivers (IDE,...) check only the class of hardware so I think there could be error on hardware that we could detect. But my point was about one of my mouse: it work good on windoze, but not on my systems (automatic backspace after click). For this reason I would like to have component tests. Anyway I think it is more an interface problem, not a really problem, and it is not the primary goal. -- to late -- I'll read and answer the bottom part of the email tomorrow ;-) ciao cate I'm not so sure that the test should be packages. The system is too different on early boot from a Debian system. In what way? Note that a Debian Live system *is* a normal Debian system; you build a Debian Live image by giving the debian-live script a bunch of packages; it then installs them into a chroot,
Re: Hardware compatibility test: draft proposal
On Fri, Aug 22, 2008 at 11:05:49AM +0100, Chris Lamb wrote: Wouter Verhelst wrote: As I mentioned in my blog[1], I kindof like the suggestion that Bdale came up with during Debconf that we write a hardware compatibility test of sorts that hardware vendors could run on their own hardware I think this is a great idea, and I generally agree with your decisions and assumptions. - Scripts should not just test for availability of hardware. Instead, they should test the actual functionality; e.g., tests for a network interface should be done by trying to DHCP off that interface, X.org drivers should try to start X and ask for input using a graphical dialog, and tests for a hard disk should be done by trying to read some data from the disk. I think this the most important paragraph of all. What I think is missing is some really concrete info on just how various hardware items would be tested. Well, that's the hard bit, isn't it? ;-) I didn't go into much detail here yet, simply because I feel this is something we'll have to figure out as we write everything. For example, in the case of ethernet adaptors, I feel that simply successfully DHCP-ing on an interface is really not an acceptable test. As an example, Etch's kernel contains various modules (such as sky2) that kinda work on today's hardware - whilst the driver would probably pass a DHCP test, the actual performance or reliability of the device is completely inadequate. Wireless devices would pose an even more difficult problem, as support for the various encryption modes that the device supports tend to be developed at different times, etc. (I'm fairly confident these comments have parallels with other hardware categories, I just give networking examples that most readers might be able to relate to) Whilst I am aware that a testsuite could never be 100% conclusive (and I'm sure you are aware too) my underlying enquiry here is trying to work out what level of confidence you are aiming for. Yes, good point. The main problem is, at this point I'm not actually all that sure of it, really. On the one hand, I want to test as much as is possible. On the other hand, that is going to get very hard very quickly, and would probably take forever to develop. We'll probably need to find a bit of a compromise here: initially, test for at least basic functionality, and test additional functionality if we find out at some point that common modern hardware has problems where basic functionality isn't available. (Hm, this didn't mean to be so negative..) Negative? What's negative? ;) - A Debian Live image will be provided that will install the 'debian-hct' package plus all packages that say 'Provides: hardware-compatibility-test' plus all their dependencies. This will be the hardware compatibility test that we can give to vendors. Please let me know if you would like any help with Live image foo. That's the final step. If/when I get there, I will. If I still need it then... -- Lo-lan-do Home is where you have to wash the dishes. -- #debian-devel, Freenode, 2004-09-22 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Hardware compatibility test: draft proposal
On Fri, Aug 22, 2008 at 12:55:34PM +0200, Giacomo Catenazzi wrote: Wouter Verhelst wrote: Hi, As I mentioned in my blog[1], I kindof like the suggestion that Bdale Yes, I find the talk very interesting. So, after more than twelve hours of boredom on an airplane and half a night of not-being-able-to-sleep-due-to-jetlag, which is certainly enough to think about this problem, I came up with the following things such a system could need: /me too - It should support a notion of what I'll call 'profiles'. A 'server' profile should check for different things than a 'desktop' or 'laptop' profile; e.g., it's usually okay if a server doesn't have graphical support or wireless drivers, while the same isn't true for a laptop. No. I don't think we should provide profiles. The check should be: complete: test all hardware that it is installed on system. If the system doesn't have the video-card or the wifi just it doesn't test it, but it write a notice. That will create ambiguity, which I think we should avoid at all cost. If you tell a user of this test that it seems to work, but you should check this and this and this output to make sure we didn't miss anything, then such a test is worthless; hardware vendors *will* miss this kind of thing. Listening the available hardware is pretty trivial (see i.e. my AutoKernConf). I'm afraid I'm not familiar with this piece of software; and as I'm offline right now, I can't look it up, either. However, I don't think checking by listing the hardware is the best way to go, in any regard. A script that checks for all hardware available in a system can't know about each and every piece of hardware out there; especially not if new interfaces are made that the script doesn't yet know about. Giving a piece of hardware a perfect score because our script failed to notice there's some unsupported wireless interface connected is not the way to go. Additionally a component test should be furnished, for component hardware vendor. (e.g. a network card vendor would test only the network card, not the whole system). I'm not sure that's very useful. A network card vendor has two options: either they design a card based on an existing chipset that has drivers already in the kernel (in which case it will probably just work, unless they use previously-unknown PCI IDs, in which case it won't), or they will create a totally new chipset for their network card, in which case it won't work. A vendor is usually aware of this fact, and giving them a test for something they already know about isn't very useful. If you disagree, of course don't let me stand in your way. Just don't expect me to work on it ;-) - The vendor should be able to influence the score of a test by explicitly stating that particular hardware isn't available. If the vendor really wants to build a laptop without wireless drivers in this day and age, then it's obviously okay if no wireless drivers were detected. If, however, the vendor is not insane, then the failure to detect a wireless chipset should clearly influence the score. See above. The score should be done only negatively, i.e. when a hardware is found but: - doesn't work properly - it is buggy (as for the BIOS: linux have some work-around for broken BIOS, but the test should alert vendor) - it requires manual configuration (partly could be a problem of Debian, but it could be also a problem in hardware: wrong identification strings/number) - requires non-free stuff - ... Yes, I agree; that's the only sane way to do it. If we don't find any issues at all, a system should get a 100% score; only when we do find some issue should the score be lower. Now, with the above in mind, and after having considered Holger's proposal to do this with Debian Live[2], I think the following generic spec should cut it, but I'm open to other suggestions at this point. It's also not very detailed yet, but since no code has been written yet, that doesn't really matter at this point. - A base package 'debian-hct' will provide a basic infrastructure for these tests to run in and an initscript that actually runs them. It will also contain some tests that are useful for /any/ system, such as do we find something that looks like a harddisk controller etc. we really want debian in package name? Should we do a more broad project? I'm not opposed to making this useful to more distributions than just Debian; however, I also think that we should provide an official Debian test that will test whether something will work with Debian Stable, rather than just a random Linux kernel plus some random software that's available on some random website. The former is something tangible; the latter isn't. - Additional packages may provide tests. Packages that do so should say 'Provides: hardware-compatibility-test' in their control file. and Depends it should on 'debian-hct', to
Re: Hardware compatibility test: draft proposal
Twas brillig at 06:24:07 20.08.2008 UTC+02 when [EMAIL PROTECTED] did gyre and gimble: [] JFYI, there's great piece of software which can be reused: http://www.inquisitor.ru/about/ -- pgpMgQtDda53n.pgp Description: PGP signature
Re: Hardware compatibility test: draft proposal
Wouter Verhelst wrote: As I mentioned in my blog[1], I kindof like the suggestion that Bdale came up with during Debconf that we write a hardware compatibility test of sorts that hardware vendors could run on their own hardware I think this is a great idea, and I generally agree with your decisions and assumptions. - Scripts should not just test for availability of hardware. Instead, they should test the actual functionality; e.g., tests for a network interface should be done by trying to DHCP off that interface, X.org drivers should try to start X and ask for input using a graphical dialog, and tests for a hard disk should be done by trying to read some data from the disk. I think this the most important paragraph of all. What I think is missing is some really concrete info on just how various hardware items would be tested. For example, in the case of ethernet adaptors, I feel that simply successfully DHCP-ing on an interface is really not an acceptable test. As an example, Etch's kernel contains various modules (such as sky2) that kinda work on today's hardware - whilst the driver would probably pass a DHCP test, the actual performance or reliability of the device is completely inadequate. Wireless devices would pose an even more difficult problem, as support for the various encryption modes that the device supports tend to be developed at different times, etc. (I'm fairly confident these comments have parallels with other hardware categories, I just give networking examples that most readers might be able to relate to) Whilst I am aware that a testsuite could never be 100% conclusive (and I'm sure you are aware too) my underlying enquiry here is trying to work out what level of confidence you are aiming for. (Hm, this didn't mean to be so negative..) - A Debian Live image will be provided that will install the 'debian-hct' package plus all packages that say 'Provides: hardware-compatibility-test' plus all their dependencies. This will be the hardware compatibility test that we can give to vendors. Please let me know if you would like any help with Live image foo. Regards, -- Chris Lamb, UK [EMAIL PROTECTED] GPG: 0x634F9A20 signature.asc Description: PGP signature
Re: Hardware compatibility test: draft proposal
Wouter Verhelst wrote: Hi, As I mentioned in my blog[1], I kindof like the suggestion that Bdale Yes, I find the talk very interesting. So, after more than twelve hours of boredom on an airplane and half a night of not-being-able-to-sleep-due-to-jetlag, which is certainly enough to think about this problem, I came up with the following things such a system could need: /me too - It should support a notion of what I'll call 'profiles'. A 'server' profile should check for different things than a 'desktop' or 'laptop' profile; e.g., it's usually okay if a server doesn't have graphical support or wireless drivers, while the same isn't true for a laptop. No. I don't think we should provide profiles. The check should be: complete: test all hardware that it is installed on system. If the system doesn't have the video-card or the wifi just it doesn't test it, but it write a notice. Listening the available hardware is pretty trivial (see i.e. my AutoKernConf). Additionally a component test should be furnished, for component hardware vendor. (e.g. a network card vendor would test only the network card, not the whole system). - The vendor should be able to influence the score of a test by explicitly stating that particular hardware isn't available. If the vendor really wants to build a laptop without wireless drivers in this day and age, then it's obviously okay if no wireless drivers were detected. If, however, the vendor is not insane, then the failure to detect a wireless chipset should clearly influence the score. See above. The score should be done only negatively, i.e. when a hardware is found but: - doesn't work properly - it is buggy (as for the BIOS: linux have some work-around for broken BIOS, but the test should alert vendor) - it requires manual configuration (partly could be a problem of Debian, but it could be also a problem in hardware: wrong identification strings/number) - requires non-free stuff - ... Now, with the above in mind, and after having considered Holger's proposal to do this with Debian Live[2], I think the following generic spec should cut it, but I'm open to other suggestions at this point. It's also not very detailed yet, but since no code has been written yet, that doesn't really matter at this point. - A base package 'debian-hct' will provide a basic infrastructure for these tests to run in and an initscript that actually runs them. It will also contain some tests that are useful for /any/ system, such as do we find something that looks like a harddisk controller etc. we really want debian in package name? Should we do a more broad project? - Additional packages may provide tests. Packages that do so should say 'Provides: hardware-compatibility-test' in their control file. and Depends it should on 'debian-hct', to have a common interface. - Tests are found in /etc/hw-compat-tests. This directory will have subdirectories, one for each of 'hard disk controller', 'wired network interfaces', 'wireless network interfaces', etc. The scripts in this directory will run in asciibetical order, so that, e.g., drivers that need firmware to be loaded can ensure this firmware is actually loaded before allowing the generic test for this class of hardware to be ran. I don't like to have it in /etc. IMHO it is better to have it in /var/lib/ or ... and copied to a root directory on the target system. - A Debian Live image will be provided that will install the 'debian-hct' package plus all packages that say 'Provides: hardware-compatibility-test' plus all their dependencies. This will be the hardware compatibility test that we can give to vendors. the BIOS testing and memory testing should be included on the basic test. I'm not so sure that the test should be packages. The system is too different on early boot from a Debian system. I think it is a lot simpler to do a separate project, using building infrastructure as d-i or debian-live. IMHO the target CD should be very easy to create and it should be complete. (but maybe allowing additional external kernel and modules). The modular thing should be put mainly on the testing development side. - Vendors who pass this test on a particular bit of hardware are allowed to advertise that fact; it might be nice to have provide them with a logo that they may use for this purpose. This should be discussed after we have comprehensive tests, and probably to LI. I don't want to see packages with logo of 10 distributions. Marketing people could choose to include on packages only few distribution, which could be negative for other distribution (and maybe for debian). ciao cate Thoughts? (Back to bed now. Jet lag, gotta love it) [1] http://grep.be/blog/en/computer/debian/hardware_test and .../hardware_test_followup [2] http://debian-live.alioth.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject
Re: Hardware compatibility test: draft proposal
Giacomo Catenazzi [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Wouter Verhelst wrote: Hi, As I mentioned in my blog[1], I kindof like the suggestion that Bdale Yes, I find the talk very interesting. So, after more than twelve hours of boredom on an airplane and half a night of not-being-able-to-sleep-due-to-jetlag, which is certainly enough to think about this problem, I came up with the following things such a system could need: /me too - It should support a notion of what I'll call 'profiles'. A 'server' profile should check for different things than a 'desktop' or 'laptop' profile; e.g., it's usually okay if a server doesn't have graphical support or wireless drivers, while the same isn't true for a laptop. No. I don't think we should provide profiles. The check should be: complete: test all hardware that it is installed on system. If the system doesn't have the video-card or the wifi just it doesn't test it, but it write a notice. Listening the available hardware is pretty trivial (see i.e. my AutoKernConf). Ok. What does the test do if it notices that a pecie of hardware exists, but cannot identify it? Should it warn about that? The long list of hardware not found may be a problem. Let's picture a modern desktop PC. No floppy drive found. No touchpad found. No webcam found. No firewire found. No modem found. (This is common enough these days, but we would definately want to test any found modem due to the proliferation of weird winmodems these days). No physics acellerator found. No PC card slot found No HD-DVD drive found No Blu-ray drive found No RAID controller found No gigabit ethernet found No EFI firmware found (This theoretical PC is not a Intel Mac.) No fingerprint scanner found. (The list could go on and on). Perhaps a few of those could be merged into other tests, like a single optical drive test, but in some of those cases it would then be important that a list of hardware that was found is printed. That way the manufacturer can be sure that both optical drives (in a two optical drive PC) were found. If there is any hardware that cannot be tested with only a single test script due to major implementation differences between all brands of hardware (and no current unified software interface), we would want to avoid having a notice printed for each possible manufacture of the device. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Hardware compatibility test: draft proposal
On Wed, Aug 20, 2008 at 06:24:07AM +0200, Wouter Verhelst wrote: So, that's probably it at this point. I should have a look at bdale's talk once it becomes available at meetings-archive.debian.net, but that doesn't yet seem to be the case. If someone who was at bdale's talk has anything to add, that'd be welcome. If someone could think of anything even without being at bdale's talk, that's welcome too, of course. http://meetings-archive.debian.net/pub/debian-meetings/2008/debconf8/high/508_hp.comgodebian.ogg Gruesse, -- Frank Lichtenheld [EMAIL PROTECTED] www: http://www.djpig.de/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Hardware compatibility test: draft proposal
[Wouter Verhelst] As I mentioned in my blog[1], I kindof like the suggestion that Bdale came up with during Debconf that we write a hardware compatibility test of sorts that hardware vendors could run on their own hardware to test whether Debian works on their system. I saw Bdales talk and I like it too. I did not spend time thinking about your proposal, but wanted to let you know how we test compatibility in a Debian Edu installation. It is done in a installation where PXE installing work, and the first step in the compatibility test is to install Debian Edu automatically with all questions preseeded. If installation work, the hard drive work. If the KDE login manager show up (kdm), the graphics card work. Next step is to log in, and if a small sound is played, the sound card is working. Start a web browser, and visit URL:http://www.skolelinux.org/. If it work, the network card is functioning. Next is 3D graphics. SelectScience Math-Stellarium from the K meny, and see if the program is snappy. If it is, accelerated OpenGL is working and the 3D stuff should work fine. Insert a USB memory stick, and see if a dialog box pop up after a few seconds to ask what should be done. If it does, USB is working. Last, run nvram-wakeup and see if the motherboard and BIOS version is supported. If it is, shutdown-at-night will work independently of Wake On LAN support. I hope a Debian compatibility test will check similar things. Happy hacking, -- Petter Reinholdtsen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Hardware compatibility test: draft proposal
On Thu, 21 Aug 2008, Petter Reinholdtsen wrote: It is done in a installation where PXE installing work, and the first step in the compatibility test is to install Debian Edu automatically with all questions preseeded. If installation work, the hard drive work. [...] Your idea is good, and a simple way to partially test things, yes. I'd add a pre-run of the Linux firmware kit, and also grep the kernel log for any ACPI strings. IMHO, one should reject outright any machines that outputs any warnings from the firmware kit, unless the vendor promises to fix it. Even warnings. These things come back to bite you later. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Hardware compatibility test: draft proposal
Hi, As I mentioned in my blog[1], I kindof like the suggestion that Bdale came up with during Debconf that we write a hardware compatibility test of sorts that hardware vendors could run on their own hardware to test whether Debian works on their system. The rationale for such a test: while most of us know how Debian works and how one should test whether their hardware actually works with Debian, it is not reasonable to expect the same of hardware vendors for /all/ GNU/Linux distributions out there. According to Bdale, currently all other major operating system vendors, including the commercial Linux distributions, already provide such a test to the hardware vendors. For this purpose, it is okay if such a test is interactive to some extent (after all, it is something that you would run on a hardware prototype in a lab, not on each and every production machine), although I'm thinking a hardware compatibility test could be useful in more cases, where it might be better if it wasn't interactive. So, after more than twelve hours of boredom on an airplane and half a night of not-being-able-to-sleep-due-to-jetlag, which is certainly enough to think about this problem, I came up with the following things such a system could need: - It should be modular. People who maintain driver packages for particular hardware may want to write additional tests that a vendor may want to run; and if this particular package supports it, the driver package maintainer may want to provide pointers to a particular package so that an inexperienced user may be able to configure their hardware after running the test themselves. - The different tests should each be able to communicate what type of hardware they test for, whether that particular piece of hardware has been found, and whether it actually works. The test of whether something works may require that network connectivity, hard disks, or other similar things are available in or to the system, as applicable. - It should support a notion of what I'll call 'profiles'. A 'server' profile should check for different things than a 'desktop' or 'laptop' profile; e.g., it's usually okay if a server doesn't have graphical support or wireless drivers, while the same isn't true for a laptop. - The vendor should be able to influence the score of a test by explicitly stating that particular hardware isn't available. If the vendor really wants to build a laptop without wireless drivers in this day and age, then it's obviously okay if no wireless drivers were detected. If, however, the vendor is not insane, then the failure to detect a wireless chipset should clearly influence the score. So, that's probably it at this point. I should have a look at bdale's talk once it becomes available at meetings-archive.debian.net, but that doesn't yet seem to be the case. If someone who was at bdale's talk has anything to add, that'd be welcome. If someone could think of anything even without being at bdale's talk, that's welcome too, of course. Now, with the above in mind, and after having considered Holger's proposal to do this with Debian Live[2], I think the following generic spec should cut it, but I'm open to other suggestions at this point. It's also not very detailed yet, but since no code has been written yet, that doesn't really matter at this point. - A base package 'debian-hct' will provide a basic infrastructure for these tests to run in and an initscript that actually runs them. It will also contain some tests that are useful for /any/ system, such as do we find something that looks like a harddisk controller etc. - Additional packages may provide tests. Packages that do so should say 'Provides: hardware-compatibility-test' in their control file. - Tests are found in /etc/hw-compat-tests. This directory will have subdirectories, one for each of 'hard disk controller', 'wired network interfaces', 'wireless network interfaces', etc. The scripts in this directory will run in asciibetical order, so that, e.g., drivers that need firmware to be loaded can ensure this firmware is actually loaded before allowing the generic test for this class of hardware to be ran. - Scripts in the subdirectories, when ran, should end their output with on a single line the letter 'S', followed by a colon, followed by two numbers separated by a '/'; the first is the score, the second the maximum possible score for this script. All output that precedes the score is redirected to a file for the user to inspect later. Scripts should make sure to avoid having a line that starts with 'S:' in this preceding output, perhaps by escaping it with a leading space or some such (this leading space will not be stripped). - If no hardware is found that would be supported by the driver this script checks for, then the second number, the maximum possible score for this script is, by definition, 0. - If the hardware being tested for can only be
