14.10.2015 21:18, Russ Allbery пишет:
In other words, this is really about available resources and perceived value, and the calculus of that is different for a volunteer project that doesn't care at all about feature checklists or other drivers that might push one towards doing the work to get the conformance test suite set up. Debian also relies heavily on upstream software maintainers to handle testing and debugging individual software components rather than doing a lot of distribution-level testing. That again isn't a judgement about utility, just a statement about resources. I'm sure Debian would get a lot out of more distribution-level testing, but it's a volunteer-maintained distribution, and someone has to care enough to work on it. And that sort of work requires sustained effort over a more extended period of time to bear fruit.
Yes, this is perfectly understood. I also have an observation that in community-driven distributions most volunteers prefer to spend their time on some "creative" work (adding new features or just building new package versions) than to relatively boring tasks such as analyzing test results (esp. keeping in mind that huge test suites tend to produce false positives from time to time). I don't know a way to motivate somebody to investigate a failure in comprehensive OLVER test suite (used in LSB) instead of building a new version of some nice game:) So no wonder that more testing activity comes from commercial companies (which do use LSB tests from time to time, as we can see in LSB bugzilla where a lot of bugs concerning test suites were filed by people from several well-known companies).
Please don't treat my previous mail as a criticism of Debian maintainers, I don't intend to tell people that they spend their free time in a wrong way when working on a really great community project. But I really think that thorough testing is underestimated in many Linux components. As you said, you heavily rely on upstream (and you are definitely not alone here), but one should remember that upstream developers of many projects are also volunteers who just don't have enough time to develop and maintain complex test suites. We all should keep this in mind when discussing quality of our products and try to improve the situation. It is often said that the strength of open code is that a lot of eyes are looking at it, but I believe that a million of volunteer eyes won't replace a good automated test (I guess a lot of people ever tried to read smb. else's Perl code would agree with this:))
-- Regards, Denis. _______________________________________________ lsb-discuss mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lsb-discuss
